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Abstract 

The  theory  of  nonzero  sun  differential  games  (NZSDG)  le  extended 
for  a claaa  of  problems  in  which  the  nonlinear  system  equations  have 
hounded  control  variables  appearing  linearly.  For  terminal  coat 
functions  this  class  of  problems  is  shown  to  exhibit  "bang-bang"  type 
control  laws  with  the  possibility  of  singular  controls.  A condition 
is  derived  to  test  for  continuity  of  the  Influence  functions  when 
controls  switch  from  a nonsingular  to  a singular  control  on  alngular 
surfaces.  Two  generalized  Corns  of  the  tranaversallty  conditions 
sre  derived  for  NZSDG  theory  extending  results  of  Dreyfus  and  Isaacs. 

NZSDG  theory  is  shown  to  be  useful  in  modeling  combat  problems 
in  which  the  goals  of  the  players  are  not  diametrically  opposed. 

A two  player  and  a three  player  penetrator-lnterceptor  problem  are 
preaented  as  a NZSDG.  Numerical  solutions  for  a totally  singular  problem 
are  carried  out  to  illustrate  application  and  a typical  solution.  A 
two  player  NZSDG  pursult-evaalon  problem  is  analyzed  in  which  the  cost 
functions  of  the  two  players  are  different  functions  of  the  terminal 
range  and  angle  off. 
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I.  Introduction 

Isaacs  [17]  introduced  differential  game  theory  in  1954.  and 
since  that  tine  many  researchers  have  investigated  the  theory  for 
application  to  conbat  problems  between  two  vehicles.  (A  fairly 
complete  reference  list  is  compiled  in  ref.  [IS]).  The  theory 
Introduced  by  Isaacs  is  a zero  sum  theory-  so  called  because  the 
goals  of  the  two  players  are  assumed  to  be  precisely  opposite. 

Many  conflict  situations,  most  notably  pursuit-evasion  type  conflicts, 
are  adequately  modeled  by  this  theory,  the  result  being  simultaneous 
"optimal"  solutions  for  each  player.  However,  there  are  aspects  of 
the  pursuit-evasion  differential  gams  that  cannot  be  modeled  by  the 
zero  sum  theory.  When  the  goals  of  the  two  players  are  not  diametri- 
cally opposed  or  if  there  are  more  than  two  vehicles  in  the  conflict, 
each  with  a different  goal,  then  a more  general  theory  is  required. 
Recent  investigations  [8,  27,  33]  of  nonzero  sum  differential  game 
(NZSDG)  theory  have  shown  this  theory  to  be  much  more  general,  and 
in  fact,  zero  sum  differential  game  and  optimal  control  theory  can 
be  considered  as  subclasses  of  NZSDG  theory.  This  NZSDG  theory  can 
be  used  to  model  two  player  combat  problems  in  which  the  goals  of 
the  players  are  not  precisely  opposite,  and  problems  in  which  there 
are  more  than  two  players. 

One  of  the  primary  obstacles  in  application  of  NZSDG  theory 
to  practical  problems  is  the  fact  that  the  Influence  functions  in  the 
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generslii sd  Euler-Lagrange  equations  and  tbs  partial  derivatives 
of  the  value  function  in  the  generalised  Haallton-Jacobl-Bellman 
partial  differential  equations  can  be  dlacontinous  when  controls 
are  discontinuous.  This  problem  does  not  occur  In  optimal  control 
problems  and  rarely  occurs  In  zero  sum  differential  game  problems 
[93.  If  these  discontinuities  do  occur  in  a problem,  the  solution 
becomes  quite  difficult  since  special  conditions  such  as  the 
Welerstrass-Erdmann  corner  conditions  must  be  employed  to  calculate 
the  discontinuities. 


pose  of  Dissertation 


The  purpose  of  this  dissertation  is  to  extend  HZSDG  theory  to 


enable  solutions  for  a class  of  combat  problems  Involving  two  or 


more  vehicles. 


The  original  work  In  thin  dissertation  consists  of:  (1)  generalising 

to  NZSDG  theory  the  transversallty  conditions  of  Isaacs'  [17]  zero  sum 
differential  game  theory  and  of  Dreyfus'  [10]  optimal  control  theory. 

This  generalization  la  desired  since  these  forms  of  the  transversallty 
conditions  are  familiar  and  easy  to  apply.  (2)  developing  a theorem 
which  testa  whether  the  Influence  functions  are  continuous  at  the 
junction  of  nonsingular  trajectories  with  singular  surfaces  for  a class 
of  terminal  cost  problems  vlth  nonlinear  state  equations  and  bounded 
linear  controls.  This  theorem  allows  one  to  test  a problem  In  the 
class  to  determine  If  the  Influence  functions  will  be  continuous  on 
singular  surfaces.  Influence  function  continuity  considerably  eases 
the  obtaining  of  solutions.  (3)  applying  the  theorem  above  and  NZSDG 
theory  to  NZSDG  penetrator-lntetceptor  problems  and  to  a two  playar 
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NZSDG  pursuit-evasion  problem. 

\ 

\ 

Guide  to  Subsequent  Chapters 

In  Chapter  II  the  mathematical  background  and  necessary  theory  Is 
presented.  Chapter  TII  defines  the  class  of  problem?  considered  In 
this  dissertation  and  the  Influence  function  continuity  theorem  is 
stated  and  proved.  Chapter  Iv  presents  a two  player  and  three  player 
NZSDG  interceptor-penetrator  problem.  The  solutions  ..nd  control  lavs 
are  characterized,  and  for  the  totally  singular  problem  numerical 
solutions  are  obtained.  Chapter  V defines  a two-aircraft  NZSDG  pursuit 
evasion  problem  In  which  the  goals  of  tne  players  are  not  precisely 
opposite.  The  problem  Is  posed  as  a fixed  terminal  time,  terminal  cost 
problem.  The  control  laws  and  singular  surfaces  are  characterized  for 
the  general  problem,  and  two  special  cases  are  examined  by  finding  some 
backward  solutions. 

The  significant  contribution  of  this  dissertation  is  to  show  that 
NZSDG  theory  can  be  employed  to  model  combat  problems  with  two  or  more 
combatants  and  that  equilibrium  solutions  can  be  obtained  from  the 
theory* s application;  also  that  the  use  of  NZSDG  theory  results  in  a 
more  general  problem  and  provides  more  flexibility  and  realism  in 
modeling  the  goa-s  of  each  combatant. 
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II.  Nongaro  Sun  Differential  Cm  Theory 

Thia  chapter  presents  the  definitions,  concepts,  end  theory  of 
a deterministic  NZSDG.  Necessary  conditions  for  an  equilibrium 
solution  and  special  conditions  on  switching  surfaces  are  presented. 

The  results  of  Case,  Ho,  Prasad,  Rageia,  Saras,  and  Starr  Ref.  [18,  14, 
IS,  16  27,  28,  11,  33]  ere  the  primary  sources  for  the  aaterlel  in  this 
chcpter. 

Problem  Formulation 

Essential  to  every  differential  game  problem  ere  three  entitles 

I.  Players 

II.  Cost  Functions 
ill . Information  Sets 

The  role  of  each  of  these  entitles  in  NZSDG  theory  is  discussed  below. 
The  state  of  the  N Players  in  a NZSDG  is  governed  by  the  vector 
differential  equation 

x ■ f (x,  t,  U)  x (tQ)  - x0  (2.1) 

where  x is  the  n-dlmenslonal  atete  vector  containing  the  state 
components  of  the  N players,  t represents  time,  and 

U - (01,  1)2,  . . .,  Ui,  . . .,  U»)  (2.2) 

where  U*  la  the  control  vector  of  the  1th  player,  and  generally  player 
l's  choice  of  0*  is  constrained  to  a constraint  set  fl*.  The  state  x 
evolves  from  the  initial  state  x0  to  some  final  state  x(tf)  where 
x(tf)  Ilea  on  an  n-dlmenslonal  terminal  manifold 

* (*(tf),  tf]  - 0 (2.3) 
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An  n-dlmenslonal  terminal  surface  in  the  n + 1 dimensional  epace  of  x 
and  t is  chosen  because  of  its  analytic  tractablllty.  The  final 
time  t«  la  free  unless  otlervlsc  specified.  The  Banner  la  •-•’itch  x 
evolves  lr  detensined  by  Eq  (2.1)  and  the  players'  choice  of  controls. 

The  coat  function  of  the  lch  player  Is 
t, 

J1  - pi  Ix(tf),  t£]  + / 1 Li  [x(t>,  t.  U]  dt 

l°  1 - 1,  ....  K (2.4) 

See  Appendix  A for  a discussion  of  meaningful  cost  functions. 

Finally,  each  player  aakea  hla  control  decisions  baaed  upon  the 
information  available  to  him.  Tor  our  purpose  Information  can  be 
placed  In  two  categories: 

(1)  knowledge  of  the  capabilities  and  goals  of  all  the  players,  and 

(2)  knowledge  of  the  state  x. 

Perfect  Information  in  category  (1)  Implies  eech  player  knows  the 
state  equation  Eq  (2.1),  the  termination  criteria  Eq  (2.3),  and  the 
cost  functions  Eq  (2.4).  The  extent  of  Information  In  category  (2) 
la  usually  represented  by  observation  equations  so  that  the  1th  player's 
state  Information  yl  is  given  by 

yi  - h1  (x,  t)  1 - 1,  ....  N (2.5) 

Each  player  selects  his  control  according  to  a rule  (control  law) 
based  on  the  observations, 

U1  • U*  (y1,  t)  1 « 1,  ....  N (2.6) 

The  function  U1  is  called  player  i's  strategy.  When  player  l's  state 
Information  la  perfect  yi  = x,  and  Eq  (2.6)  becomes 

u1  » U1  (x,  t)  (2.7) 
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Eq  (1.7)  1*  called  a closed-loop  control  law.  On  the  other  hind , 
it  no  atate  Information  except  x0  la  known  hy  player  1,  then  y*  3 xg 
and  tha  control  law 

u1  - U1  (x0,  t0.  t)  (2.8) 

Is  called  an  open-loop  control  law.  These  two  control  laws  will 
be  considered  In  this  dissertation  along  with  another  control  law  to  be 
discussed  later  called  an  "open- loop  feedback  control  law". 

Solution  Concepts 

Three  solution  concepts  exist  In  the  N2SDG  theory:  (1)  equilibrium, 

(2)  mini-max  (security),  and  (3)  noninferior  (Pareto  optimal).  This 
dissertation  Is  concerned  only  with  the  equilibrium  solution;  however, 
the  two  remaining  solutions  will  be  discussed  briefly. 

Equilibrium  Solution 

In  the  equilibrium  solution  each  player's  goal  la  to  minimise  hie 
own  coat  function,  thus  making  this  solution  a noncooperative  one. 

The  following  mathematical  definitions  and  equations  define  the 
solution.  Define  the  set  of  equilibrium  strategies  U*  and  the  set 
(U*;  l'l)  as 

U*  5 (U1* U**)  (2.9) 


(u*,  ul)  = (u1* u1'1*.  ul , u,+i:  . . .,  UN*)  < 

For  Che  NZSDG  formulation  in  Eq*  (2.1)  - (2.8),  if  thero  exist*  a 
strategy  sec  U*  such  that 

J1  (U*)  « jl  (U*;  U1)  1 • 1,  ....  N ( 

then  U*  is  said  to  be  an  equilibrium  strategy.  The  trajectory 


x*  - X (V  tQ.  U*) 
is  called  an  equilibrium  trajectory. 
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Mini-Max  (Security)  Solution 

On*  of  the  categoric*  of  information  discussed  zar'iar  Is  this 
chapter  la  knowledge  of  all  players ' coat  functions,  It  thin 
information  la  not  available  to  each  player,  then  a p layer  may  adept 
a conaervatlve  viewpoint  and  aaauaa  all  other  players  are  opposing 
hie.  Such  a atrategy  la  called  a mini-max  (security)  strategy.  Starr 
[33]  points  out  that  this  solution  is  equivalent  to  solving  M t wo 
player  zero-sun  games  wherein  for  the  le^  gaae  player  1 tmleeta  a 
atrategy  to  minimize  J1  while  all  other  playera  select  strategies  to 


maximize  J1.  Once  player  1 has  eolved  his  zero  sum  g*at  (1  - 1, 


to  determine  his  etrategy,  the  N strategies  era  employed.  According 
to  Starr  [33],  the  resulting  trajectory  la  generally  surprising  to 
each  player  because  of  ;ta  conservative  approach  taken  by  each. 

For  e further  discussion  of  this  solution,  consult  References  [27,  28, 
33,  35]. 

Nonlnfarlor  (Pareto  Optimal)  Solution 

If  a negotiated  or  cooperative  solution  can  ha  agreed  upon  by 
all  the  players  (such  sa  in  an  economic  situation),  then  all  costs  arc 
leas  than  or  equal  to  the  corresponding  costs  of  the  equilibrium 
solution.  A noninferior  solution  has  the  property  that  any  other 
solution  which  gives  a better  result  for  one  player  also  gives  a worse 
result  for  another  player.  Any  negotiated  solution  should  be  chosen 
from  the  set  of  nonlnfarlor  solutions  [35].  Consult  References  [27,  28, 
33,  3S]  for  a further  discussion  of  this  solution  concept. 

Necessary  Conditions  for  tha  Equilibrium  Solution 

Bellman1!  Dynamic  Programing  (or  principle  of  optimality)  and 
Pontryagln's  minimum  principle  can  ba  generalized  to  the  N2SDG  problem 
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to  give  two  different  necessary  conditions  for  an  equilibrium  solution. 
Bellman *s  principle  results  in  a set  of  generalized  Hamil ton- Jacobi- 
bellman  (HJB)  partial  differential  equations  which  the  equilibrium 
solution  must  satisfy.  Furthermore,  Bellman’s  principle  requres  the 
assumption  of  perfect  information  [27]  so  that  the  control  laws  are 
closed-loop  laws  as  in  Eq  (2.7).  Pontryagin’s  minimum  principle  on  the 
other  hand  is  usually  associated  with  open** loop  control  laws  as  in  Eq  (2.8), 
and  this  principle  leads  to  a set  of  generalized  Euler-Lagrange  equations 
(called  influence  function  equations  in  this  dissertation)  which  together 
with  the  state  equation  Eq  (2.1)  are  the  '’Characteristic"  equations  for 
the  HJB  partial  differential  equations.  Thus  the  HJB  equations  constitute 
c much  more  general  necessary  condition. 

In  optimal  control  and  zero  3um  differential  game  problems  the  state 
information  available  to  the  players  has  no  effect  on  the  form  of  the 
influence  function  equations,  but  the  state  information  assumed  in  a NZSDG 
problem  can  have  a marked  effect  on  the  form  of  these  equations.  This 
effect  will  be  made  clear  in  the  following  sections.  Because  the  HJB 
equations  are  seldom  solvable  except  for  unconstrained  quadratic  cost 
linear  dynamics  problems,  all  solutions  are  generally  obtained  from  the 
influence  function  equations.  The  main  utility  of  the  HJB  equations  is  to 
verify  that  an  equilibrium  solution  candidate  obtained  from  the  influence 
function  and  state  equations  satisfies  the  HJB  equations.  This  satisfaction 
of  the  HJB  equations  is  another  necessary  condition  which  an  equilibrium 
solution  must  meet. 
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Hamllton-Jacobl-Ballna n Partial  Differential  Equations 

Consider  Che  equilibrium  strategy  U*  of  Eq  (2.11).  Define  the 
value  W*  for  player  1 to  be  the  cost  of  player  1 on  an  equilibrium 
trajectory  with  the  general  lntlal  point  (x,  r), 

W 1 (x,  t)  5 p1  Jx*  <t£),  tfJ  + /tf  L1  [x*  (r),  t,  D*]  dr 


Kestricting  U*  to  be  piecewise  continuous  we  see  that  U*  Is 
contiguous  and  plecevlse  differentiable*  The  values  can  be  shown 
to  satisfy  the  -c.w^led  systen  of  partial  differential  equations  hereafter 
referred  to  as  the  HJB  equations  [27] , 


W1.  - - nln  H1  (x,  W1  , t,  B*j  U1) 

s i * 


1-1 N 


H1  S L1  (x,  t,  0)  + W1^  f (x,t,U)  (2 

with  the  boundary  condition  given  on  the  terminal  manifold 

W1  (x  (tf),  tf]  » p1  [x  (t{),  tf]  (2 

If  the  minimization  operation  In  Eq  (2. IP)  la  carried  out  subject 
to  all  constraints,  the  equilibrium  controls  U**  are  found  to  depend 
functionally  on  W*  , x and  t 


U1*  - U1*  (W1  . x,  t) 


1-1 H 


Substituting  Eq  (2.17)  Into  Eq  (2.14)  we  obtain  another  form  for 

the  HJB  equations  — a coupled  nonlinear  systen  of  partial  differential 


equations 


V*t  (x,  t)  - - H1  (x,  t,  (W^)] 

1,  J - 1.  . 

with  the  boundary  condition  given  by  Eq  (2.16). 
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The  HJB  equation*  are  defined  except  on  surfaces  in  the  state  space 
on  vhich  the  controls  are  discontinuous.  For  a rigorous  development 
of  the  HJB  equation  see  Prasad  [27]  and  Saras  [31]. 

The  advantage  of  a solution  to  the  HJB  equations  Is  clear,  for 
if  an  analytic  expression  Is  available  for  W*(x,  t)  and  Wx*  then 
from  Eq  (2.17)  one  sees  that  a closed  loop  control  lav  Is  obtained. 
Solving  for  the  values  W*  equivalent  to  finding  a field  of 
equilibrium  solutions.  Unfortunately,  one  is  generally  required  to 
settle  for  less  if  the  problem  dynamics  are  constrained  or  nonlinear. 
The  solution  of  the  HJB  equations  by  the  method  of  characteristics 
[10]  Is  the  alternative.  The  characteristic  differential  equations 
for  Eq  (2.18)  turn  out  to  be  the  state  equation  Eq  (2.1)  and  the 
influence  function  equations  from  Pontryagln's  minimum  principle. 
Influence  Function  Differential  Equations 


Assume  that  the  information  set  of  each  player  is  perfect  vhich 
Implies  a closed- loop  control  lav  for  each  player;  then  the  necessary 
conditions  vhich  must  be  satisfied  on  an  equilibrium  trajectory  are  the 


state  equation 

x • f (x,  t,  U*) 

and  the  Influence  function  equations 


x (t  ) * X. 
' o o 


- <H*  + I H1,,.  uJ  )* 


H1  - L1  (x,  t,  U)  + X1  f <x,  t,  U) 


and  U Is  the  admissible  minimizing  control  in  the  equation 
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H1*  - min  H1  (x,  X1,  t,  0*  ; 01)  (2.22) 

D1 

The  eumatlon  tern  in  zq  (2.20)  is  absent  in  optlnsl  control  and  sero 
sun  differential  gaae  problems,  and  as  we  will  see  in  later  chapters, 
varies  according  to  the  information  set  assumed.  For  example,  if 
perfect  observations  are  assumed  as  in  Eq  (2.7),  D^x  is  generally 
nonzero  in  Eq  (2.20);  however,  if  no  observation  other  than  initial 
conditions  are  assumed  ar  in  Eq  (2.8)  then  U^x  is  identically  zero. 

The  Influence  function  l1  is  identical  to  the  partial  derivative  of  the 
value  Wx^,  when  is  evaluated  along  an  equilibrium  trajectory.  The 
essential  difference  between  1*  and  is  that  A*  is  a function  of 
time  only  and  is  s function  of  both  state  x and  t regarded  as 
Instantaneous  initial  conditions.  Mathematically  the  relationship 
may  be  written 

A1  (t)  - W^1  [x*  (t),  t]  (2.22) 

where  x*  (t)  evolves  according  to  the  state  equation  Eq  (2.19). 

It  is  important  to  note  that  the  influence  equations  are  valid  over 
the  same  regions  as  the  HJB  equations  and  solutions  of  the  Influence 
function  equations  and  state  equation  must  satisfy  the  HJB  equations. 

The  influence  function  equations  are  veil  defined  except  on  surfaces 
In  the  state  space  where  the  controls  U*  are  discontinuous.  Appendix  B 
presents  a formal  derivation  of  the  influence  function  equations  from 
the  HJB  equations. 

The  boundary  conditions  for  the  influence  function  equations 
Eq  (2.20)  are  specified  in  terms  of  the  state  and  time  on  the  terminal 
manifold,  and  these  boundary  conditions  are  generally  called  transversallty 
conditions.  Several  forms  of  the  transversallty  conditions  exist; 
however  in  this  dissertation  we  will  use  a generalised  form  of  Dreyfus's 
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cransv.rsallty  condition  because  of  its  eaae  of  application.  The 
generalisation  of  the  Dreyfua  [10]  font  of  the  traneveraallty 
conditions  along  with  a generalisation  of  the  familiar  Isaaca  [17] 
form  are  presented  in  Appendix  C.  Alao  in  Appendix  C la  presented 
the  more  general  Berkovitx  fora  [5]  due  to  Sanaa  [31].  The  Isaacs 
fora  and  Berkovlts  form  ere  presented  since  both  are  in  general  use. 

The  Dreyfus  form  requires  a terminal  coat  function  of  the  form 

Ji  - (x  (tf).  tf]  i - 1,  ...»  N (2.24 

so  that  L*  ■ 0 in  Eq  (2.4).  This  is  not  overly  restrictive 
since  every  problem  with  a cost  function  containing  an  integral  can  be 
easily  converted  to  an  equivalent  terminal  cost  problem  [17].  See 
Appendix  C for  the  details  of  this  conversion. 

The  Dreyfus  form  of  the  transversality  conditions  is 

l1  (tf)  - [♦$  - (i1/*)  * ] 


where  4 and  are  total  time  derivatives. 

Since  the  state  equation  boundary  conditions  are  specified  at  tQ 
and  the  Influence  function  boundary  conditions  are  specified  at  t^, 
a two  point  boundary  value  problem  (TPBVP)  results.  Assuming  that  a 
solution  exists,  ve  see  that  solving  the  TPBVP  is  equivalent  to  finding 
a candidate  for  the  equilibrium  solution  to  the  original  NZSDG;  hence 
the  NZSDG  problem  has  been  transformed  into  an  equivalent  TPBVP. 
Solutions  to  the  TPBVP  are  called  candidates  for  an  equilibrium 
solution  since  the  TPBVP  is  a necessary  but  not  sufficient  condition. 
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Switching  Surface* 

When  we  attempt  to  eolve  the  TPBVP  consisting  of  the  atete  and 
Influence  function  equations  Eqe  (2.19)  end  (2.20),  we  often  find  that 
the  equlllbriua  controli  are  dleeontlnuoua,  especially  If  there  are 
control  constralnte  present.  These  discontinuities  In  control 
occur  on  switching  surfaces  In  the  state-tlae  space,  the  surfaces 
being  classified  as  transition,  singular,  dispersal  and  abnormal. 1 
The  HJB  and  Influence  function  equations  are  generally  not  defined  on 
these  switching  surfaces,  and  the  partial  derivatives  U1  , W*t 
and  the  Influence  functions  X1  can  be  discontinuous  when  the 
trajectory  crosses  or  enters  these  switching  surfaces. 

Special  conditions  must  be  satisfied  on  these  surfaces  which 
serve  to  continue  solutions  across  or  along  the  surfaces.  The 
following  sections  describe  the  surfaces  and  give  conditions  for 
their  construction. 

Transition  Surface 

The  transition  surface  Is  one  on  which  the  generalized  Welerstravr - 
Erdmann  comer  condition  holds  [28].  Let  (+)  and  (-)  denote  one-sided 
limits  on  respective  aides  of  the  transition  surface,  then  the  following 
corner  condition  holds  [28],  provided  that  the  trajectory  Is  not 
parallel  to  the  surface: 


1.  Thu  names  "transition"  and  "dispersal"  are  due  to  Isaacs  [17]  while 
"abnormal"  surfaces  are  surfaces  like  Isaacs'  "barrier”.  Abnormal 
surfaces  are  discussed  In  Kef.  [27], 
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(Hi+  . Hi~)  dt  . (*i+  _ x1”)  dx  - 0 


The  variations  dx  and  dt  are  arbitrary  variation!  on  the  tranaltlon 
surface.  If  the  tranaltlon  aurface  la  parametrized,  say  by  the  n - 
dlaenslonal  parameter  e where 
o = (o^,  02,  • . • ■ on) 

such  that  the  aurface  la  described  by  the  parametric  equations 
t - T (o)  x - x (o) 
then  Eq  (2.26)  takes  the  form  [28] 

(Hi+  - Hi-)  3T/3 o - (Xi+  - X1-)  3x/3o  - 0 (2.27) 

1 - 1,  ....  N 

In  Reference  [28]  it  is  proven  that  if  ail  controls  except  U**  are 
continuous  on  player  i's  tranaltlon  surface  and  the  trajectory  is  not 
parallel  to  the  surface,  then  X1,  the  1th  players  Influence  function 
vector.  Is  continuous  on  the  surface  (remember  that  the  Influence 
functions  for  the  other  players  x3 , 1 y*  j , may  be  discontinuous) . 
Singular  Surface 

Singular  surfaces  consist  of  a family  of  trajectories  on  which  for 
at  least  one  player,  say  the  1th,  a control  component  Is  on  the 

interior  of  Its  constraint  set  for  a nonzero  time  Interval  and  the 


coefficient  of  0 in  the  Hamiltonian  function  Is  identically  zero, 


1 


DS/MC/71-3 


on  this  tine  Interval.  The  aoet  coskq  and  laportant  case  occurs 
vhen  the  control  appears  linearly  In  the  state  equation  and  the 
Haalltonian.  We  will  consider  only  this  linear  case.  For  this  case 
Eq  (2.28)  does  not  suffice  to  determine  the  minimizing  (singular) 
control  since  U1^  does  not  appear  In  Eq  (2.28).  A higher  order 
necessary  condition  called  the  generalized  Legendre  Clebshe  condition 
[10,  29]  la  required  to  test  the  singular  control  which  la  generally 
deterained  by  Eq  (2.29)  below.  Although  the  above  references 
developed  the  Legendre-Clebah  condition  only  for  optlaal  control  probleaa 
the  result  has  been  extended  to  zero  sub  differential  gaaea  [1]  and  to 
NZSDG  [28].  Since  H1  is  Identically  zero  on  the  singular  arc,  so  are 

US 

all  of  Its  tlae  derivatives,  thus 

H*  ■ H1  «...  - 0 (2.29) 

u i “j 

Successive  differentiation  as  In  Eq  (2.29)  with  substitution  of  the 
state  and  Influence  function  equations  for  player  1 gem  .'ally  results  In 
an  equation  containing  explicitly.  Robbins  [29]  shows  that  If 

appears  at  all  In  Eq  (2.29)  It  first  appears  explicitly  In  an  even 
tlae  derivative  of  H*  . Not  only  does  Eq  (2.29)  often  provide  an 


equation  which  determines  U^,  but  It  also  provides  useful  relations 
among  the  state  and  Influence  function  variables  which  must  hold  along 
the  singular  arc.  The  necessary  condition  for  a singular  control 
U1J  to  be  an  adalsslble  equilibrium  control  is  the  satisfaction  of 
the  Inequality 

(-1)  k 3/3U*  [ d2k/dt2k  H1  ] > 0 (2.30 
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Hobbins  129)  call*  the  quantity  2k  the  "degree"  of  the  aingularlty 


d2k/dt2k  H1  - 0 U.31) 

Ulj  1 

la  the  equation  In  which  U j first  appears  explicitly. 

It  la  possible  that  more  than  one  control  component  la  singular 
on  the  same  time  Interval.  In  this  case  Eq  (2.30)  la  applied  to  each 
control  component.  If  one  of  the  control  components  say  j has  a 
lover  degree  of  aingularlty.  when  firat  appears  In  Eq  (2.30)  the 
functional  relationship  between  U1j  and  the  state  and  Influence 
function  variables  is  determined.  This  result  is  substituted  back 
into  Eq  (2.30)  to  eliminate  U1^  from  the  equation  then  successive 
differentiation  is  again  employed  to  find  the  control  component  with  the 
next  higher  degree  of  singularity. 

Jacobson  and  Speyer  [19,  20,  32],  and  McDannell  and  Powers  [23] 
have  derived  new  necessary  conditions  and  new  sufficient  conditions 
for  totally  singular  optimal  control  problems;  however,  it  is  not 
known  whether  these  conditions  can  be  readily  extended  to  zero  sum 
and  nonzero  sum  differential  games. 

Dispersal  Surface 

K dispersal  surface  for  player  i is  a surface  from  which  player  i 
can  play  more  than  one  equilibrium  strategy  without  changing  his  cost. 
Examples  of  this  type  of  surface  are  found  in  Isaacs'  Homocidal  Chauffer 
Game  and  Game  of  Two  Cars  [17].  The  following  equation  characterizes 
the  dispersal  surface  for  player  1 [27,  28]: 

H1  (x,  l1,  01,  t)  dt  - X1  dx  - H1  (x,  X1*,  U^,  t)dt  (2.32) 


F 


DS/MC/71-3 


where  p la  the  number  of  equivalent  strategies  and  k lndlcatca 
tha  k**1  equivalent  strategy.  dx  and  dt  ara  arbitrary  variation  on 
tha  dlaperaal  aurfaee.  Tha  dlapersal  surface  can  occur  in 
purault-ovaalon  problems.  A typical  altuatlon  illuatratlng  tha 
nature  of  tha  aurfaee  occura  whan  player  1 haa  the  option  of  aaklng 
an  equilibrium  control  choice  of  either  a turn  right  or  a turn  left. 

Aa  lta  name  "dlaperaal  aurfaee"  Implies,  trajectories  generally 
diverge  from  tha  surface. 

Vbnormal  Surface 

In  this  dissertation  normal  problems  have  been  assumed;  however, 
examples  of  abnormal  surfaces  appear  in  the  literature  and  need  to  be 
mentioned.  The  main  examples  of  such  surfaces  are  Isaacs'  semlpermeable 
surfaces  with  the  barrier  surface  a special  case  [17].  Prasad  [27] 
points  out  that  another  abnormal  surface  la  Isaacs'  Equivocal  Surface 
which  Is  a dispersal  surface  for  one  player  and  a singular  surface  for 
the  other.  The  reader  Interested  in  abnormal  surfaces  Is  referred  to 
Kef  [27]. 


This  chapter  haa  presented  the  background  and  theory  of  NZSDG 
to  acquaint  the  reader  with  the  topic.  The  two  approaches  to  necessary 
conditions  provide  the  HJB  equations  and  the  influence  function  equations. 
In  practice,  for  nonlinear  or  constrained  problems  It  Is  usually  required 
to  solve  the  TPBVI  consisting  of  the  Influence  function  equations  and 
state  equation  in  order  to  obtain  equilibrium  solutions.  The  conditions 
necessary  to  construct  the  various  switching  surfaces  which  are  required 
In  solving  the  TPBVP  were  presented.  It  should  be  noted  however,  that 
these  conditions  are  generally  difficult  to  apply. 


17 


Di/MC/71-3 


[ 

i 


i 


XII.  A Class  of  Monllnear  MZSDG  Problems  with  Control*  Appearing  Linearly 

A broad  claaa  of  differential  game  probleu  Involves  a nonlinear 
dynamical  syetem  with  controls  appearing  linearly  In  the  state  equation. 
Tbit  chapter  treats  the  terminal  coat  NZSDG  which  results  from  the 
above  type  of  dynamics  with  constrained  controls.  The  following  sections 
define  the  problem  class,  formulate  the  differential  game,  HJB  equations, 
influence  function  equations,  TPBVP , define  the  control  switching 
functions,  and  present  conditions  under  which  the  Influence  functions  are 
continuous  on  singular  surfaces.  The  singular  surface  occurs  frequently 
in  many  problems  and  is  often  the  only  switching  surface  Involved  in  the 
problem;  therefore,  it  Is  Important  to  know  the  behavior  of  the  Influence 
functions  on  this  surface.  If  the  Influence  functions  sre  continuous  at 
the  junction  of  a trajectory  and  a singular  surface  the  solution  of  the 
TPBVP  la  considerably  eaaler. 

Chapters  IV  and  V are  devoted  to  two  examples  from  this  class  of 
problems . 

Problem  Statement 

Por  the  class  of  problems  considered  in  this  chapter  the  state 
equation  Is 


• N 1 

x • g (x)  + I AJ  (x)  uJ  - f (x,  U) 

J - 1 

* <t0>  - *0 


(3.1) 


The  control  variable,  U*  corresponding  to  the  1th  player  Is  a scalar 
and  Is  constrained  such  chat  the  following  inequality  holds 


ill1!  < 1 


1.  • 


.,  N 


(3.2) 


i 

5 

i 
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(Thu  problem  can  b«  generalized  for  vector  ccntrola  with  the  major 
effect  being  notetlonel  complexity).  The  state  vector  x contains  the 
state  components  of  the  N players. 

The  goal  of  each  player  la  to  choose  an  admissible  control  strategy 
D1*  such  that  hie  own  terminal  cost  function 

J1  - [x  (tf),  tf)]  (3.3) 

is  minimised,  r.  (tf)  and  tf  satisfy  the  termination  criteria 

* I x (t£),  t{]  - 0 (2.3) 

He  assume  perfect  Information  which  Implies  closed-loop  strategies 
for  each  player;  however,  It  will  be  shown  for  this  class  of 
problems  that  the  closed-loop  and  open-loop  control  laws  will  result 
In  the  same  Influence  function  equations  hence  the  same  equilibrium 
solutions. 

Formulation  of  the  HJB  Equations,  Influence  Function  Equations  and  the  TPBVP 


He  can  formulate  the  HJB  equations, 

H*  (x,  t)  - - min  H1  (x,  t,  H*.  U*;  U1)  (2.14) 

Ul  1-1 N 

H1  [x  (t£),  t£]  - p1  (x  (t£>,  tfJ  (2.16) 


1-1 N 


where  H is  given  by 

H1  - H1  f - H1.  ( g (x)  + £ AJ  (x)  U3]  (3. 

* i - 1 

The  admissible  U**  which  minimizes  H1  In  Eq  (3.4)  (holding  the  other 
1* 

controls  U , j d 1,  at  their  equilibrium  values)  Is  (by  Inspection  of 
Sq  (3.4)) 

U1*  - - sgn  (Hxi  A1)  (3.5 
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where 


sgn  (y)  ■ 1 y > 0 

- 1 y < 0 


0.6) 


1 .1 


Eq  (3.3)  holes  except  when  the  argument  W A satisfies  the  equation. 


t 


i, 

f 


W1^  A1  - 0 (3.7) 

1* 

over  a nonzero  time  interval.  In  this  event  it  is  possible  for  U to  be 

singular,  and  the  necessary  conditions  given  by  Eqs  (2.28)  through  (2.31) 

i* 

must  be  applied  to  determine  the  control  U and  test  its  admissibility 
as  a candidate  for  an  equilibrium  control. 

We  can  now  write  the  HJB  equations  fsr  this  problem  in  the  form 

W1,  - - W1  [g  (x)  + l Aj  UJ*]  (3.8) 

* * J-l 

with  the  boundary  condition  given  by  Eq  (2.16).  Eq  (3.8)  is  a system  of 
first  order  nonlinear  coupled  partial  differential  equations  where  the 
equations  are  coupled  through  the  last  term  of  the  r.h.a..  This  equation 
cannot,  in  general,  be  solved  in  closed  form;  however,  it  is  an  additional 
necessary  condition  since  the  state  and  Influence  function  equations  are 
the  characteristic  equations  for  the  HJB  equations  and  therefore  mist 
satisfy  it. 

The  Influence  function  equations  for  this  class  are 

X1  - - A1  ( g + Z A^  u-**  + Z A-*  u-**  ) 

* j-l  X j-l  X 

i - 1 N (3.9) 

The  terminal  boundary  conditions  for  Eq  (3.9)  are  given  by  Eq  (2.25). 

In  Eq  (3.9)  U^*  is  given  by 

UJ*  - - sgn  (AJ  Aj)  (3.10) 

or  In  the  event  the  argument  V'  satisfies  the  equation 

XJ  AJ  - 0 (3.11) 

over  a nonzero  time  Interval,  the  control  1^*  may  be  singular  and 
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must  b«  determined  by  the  necessary  conditions,  Eqe  (2.28)  through 
(2.31).  In  Eq  (3.9)  ii  J - 1.  X*  A1  U**  Is  Identically  itro 
(sea  Appendix  B) . In  any  event,  the  expression  Ux^*  In  the  lest 
term  In  the  r.h.s.  of  Eq  (3.9)  la  either  Identically  xero  In  regions 
where  the  controls  are  constant  or  undefined  on  switching  surfaces. 

In  solving  the  Influence  function  equations,  the  special  conditions 
for  the  various  surfaces  outlined  in  Chapter  II  must  be  used  to 
continue  solutions  across  the  svltchlng  surfaces.  In  the  event  a 
trajectory  lies  In  a switching  aurfaca  such  as  In  the  case  of  a 
singular  control  segment,  special  arguments  to  be  discussed  later  In 
this  chapter  must  be  employed. 

The  equation  for  the  Influence  functions  holding  between  switching 
surfaces,  can  now  be  written 

i1  - - l1  (gx  + I AJ  u^*  ) 1-1 N (3.12) 

j - 1 

Control  Laws 

Eqs  (3.5)  and  (3.10)  defining  the  control  U'  are  known  as  a 
"bang-bang"  control  law,  so  that  the  controls  for  this  class  of  problems 
are  "bang-bang"  with  the  possibility  of  singular  controls  when  Eqs  (3.7), 
or  (3.11)  holds.  The  development  thus  far  In  this  chapter  assumes 
perfect  Information,  buc  now  let  this  requirement  be  relaxed.  Suppose 
r.o  state  observations  except  x (tQ)  are  permitted.  Then,  according  to 
Eq  (7.8), 

ui  - U1  <xQ>  t0,  t)  (2.8) 

Hence  Ux*  in  Che  influence  function  equation*  Eq  (3.9)  1*  Identically  zero 
and  the  influence  function  equation*  for  the  open-loop  control  law 
problem  become 

i1  - - l1  (gx  + I AJ  uJ*  ) 1-1 N (3.13) 

i -*1 
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which  aie  identical  to  the  closed-loop  influence  function  equations 
Eq  (3.12)  for  this  class.  Thus  the  open-loop  trajectory  is  identical  to 
the  closed-loop  trajectory  for  a given  set  of  initial  conditions.  This 
identity  is  caused  by  the  control  constraints  Eq  (3.2)  and  the  linearity 
of  the  controls  in  the  state  equation  Eq  (3.1)  and  is  not  dependent  on 
the  information  pattern.  Because  the  open-loop  and  closed-loop  control 
laws  result  m identical  trajectories,  in  this  class  of  problems  a 
sampled  data  feedback  solution  (i.e.,  an  open-loop  solution  using  each 
new  sample  point  as  an  updated  initial  condition)  approaches  the  closed- 
loop  solution  in  the  limit  where  the  sample  interval  *3  allowed  to 
approach  zero.  This  limiting  behavior  is  not  true  in  the  general  NZSDG 
since  Starr  [33 j has  shown  that  the  so  called  open-loop  feedback  solution 
(where  the  initial  conditions  are  updated  instantaneously  along  the 
trajectory)  and  the  closed-loop  solution  are  different  for  linear  prob- 
lems with  no  control  constraints. 

Conditions  for  Influence  Function  Continuity  Along  Singular  Surfaces 

In  higher  dimensional  problems  ( n >_  3 ) singular  surfaces  which 
Is.acs  calls  ’‘universal"  surfaces  [17]  occur  frequently  when  bounded 
controls  appear  linearly  in  the  state  equation  as  in  the  class  considered 
in  this  chapter.  The  singular  surface  is  characterized  by  the  fact  that 
trajectories  r.nter  the  surface  from  either  side  then  proceed  along  the 
surface  itself.  Letting  M denote  the  singular  surface,  the  situation 
in  two  dimensions  is  depicted  in  Fig.  1. 

From  Chapter  II  we  know  that  the  HJB  equations  are  not  necessarily 
defined  on  switching  surfaces  (which  includes  the  singular  surface  H, 
but  the  HJB  equations  are  valid  on  either  side  of  H if  we  regard  the 
partial  derivatives  in  Eq  (3.8)  as  one  sided.  If  we  can  show  that  the 


I 
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Fig.  1.  Singular  Surface 


partial  derivatives,  WAX  and  W*t  are  continuous  on  M,  then  we  do  not 
need  to  restrict  these  partials  to  be  one-sided,  and  we  can  conclude 
that  the  HJB  equations,  Eq  (3.8),  and  therefore  the  influence  function 
equations  Eq  (3.S)  are  defined  on  M. 

Hence  we  establish  the  following  theorem: 

Given  toe  NZSDG  with  the  state  equation 

x ■ g (x)  (x)  irf  x ^co)  * x0  (3«1) 

I uJ|  < 1 

with  the  terminal  manifold  given  by 

<h  [ * (tf),  tf  ] . 0 (2.3) 

and  cost  functions  given  by 

jl  - ♦*[  x (tf),  tf  ] (3.3) 
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If  the  following  conditions  are  met  then  for  player  k,  Wkx  and  Vrt  are 
continuous  on  M: 

i.  3<x)  and  A^  (x),  j - 1,  . . •.  , N,  are  continuous  vector 
functions  of  x and  linearly  independent  for  all  v. 
it.  H is  a singular  surface  of  the  control  U1  where  M can  be  entered 
from  either  side  by  employing  U*-*  “ +1  on  one  cide  of  M or 
U1*  - -1  on  the  other  side  of  H.  Let  M*  be  the  side  of  M 
corresponding  to  U**  - +1  and  be  the  side  of  M corresponding 
to  U1*  - -1. 

lii.  The  controls  U-3*,  j t i,  are  constant  M* 

iv.  On  M the  term  Vkx  A1  (where  Wkx  is  considered  to  be  a one  sided 
derivative)  is  identically  zero, 

A1  - 0 Wi  (3.14) 

v.  The  inner  product  B (x)  A*  is  not  a constant  over  any 
nonzero  time  interval;  B (x)  is  defined  below. 


The  HJB  equations  for  player  k are 
N 

Wk  . -Wk  g (x)  - E Wk  A^  U-1*  k - 1, 

* * j-1 


N (3.8) 


Eq  (3.8)  is  detined  on  either  side  of  H but  not  on  H itself,  therefore 
the  partial  derivatives  Wk^  and  Wkt  must  be  com.lt  ed  as  one-sided 
on  M*-  and  M".  Designate  Wk+X  and  Wk+t  as  one  sideu  partial  derivatives 
on  H*  and  Wk_x  and  Wk't  as  one-sided  partial  derivatives  on  K . 

On  M,  which  is  a singular  surface  for  player  i,  the  switching 
function  W1  Ai-  is  identically  zero  (W1  is  defined  on  M (27,  28]) 

X * 

Vi  A1  » 0 (3.15) 

x 

From  Eq  (3.8)  and  condition  lv.  above  we  can  write  the  equations 
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Wk+X  Ai+  - Hk-X  Ai_  - 0 


(3.16) 


Wk+,  + Wk+  [ g (x)+  + I aJ+  uJ*+] 
3-1 


Wk-t  + W^  [ g(x)' 


i A^-  u3*-j  - o 
>1 


Since  g (x)  end  A3  (x),  J - 1,  . . . , N,  ere  continuous  in  x,end  u3*, 
j |l  i,  ie  conetent  In  the  neighborhood  of  H under  conelderetlon,  by 
continuity  arguments 

A3+  - a1~  - A3  (3.17) 


B (x)  5 g 

(x)+ 

+ 

N 

z 

aJ+ 

3*1 

3-1 

- g 

(x)- 

+ 

N 

a3- 

ll3*- 

31*1 

3-1 

• g 

(x) 

+ 

N 

Z 

a3  u^* 

3W 

3-1 


(3.18) 


B (x)  la  the  tine  derlvetlve  of  the  equilibrium  etete  vector  except 
the  conponent  containing  U1.  By  assunptlon  1.  B (x)  end 
*3  (x),  3 » 1,  . . . , N,  are  linearly  independent  vectors. 

From  Eqs  (3. 1C),  (3.17),  and  (3.18)  ve  can  now  write  the  equations 
(Wk+X  - Wk"x)  A1  - 0 k 1*  1 (3.19) 

- (Wk+t  - Hk_t)  « (Wk+y  ~ Wk-x)  B (x)  (3.20) 

Now  Vfk+t  and  Wk"t  are  constants  on  an  equlllbrlun  trajectory  along  H 
s Ince  the  state  equation  Is  autonomous  and  the  problem  has  terminal  cost 
functions;  hence  Eq  (3.20)  becomes 


1 

Jtt 

3 

f 

) 

} 

i 

-d 

i 

i 


i i 


Ck  - (wk+x  - wk_x)  B (x)  k * 1 (3.21) 

Returning  to  Eq  (3.19)  one  of  two  conditions  exist;  either  (a), 

(Wk+X  - Hk“x)  « 0,  or  (b),  (Wk+X  - Wk_x)  Is  orthogonal  to  A*.  Condition 
(a)  together  with  Eq  (3.20)  Imply  Sk+X  » Vk*x  anJ  Wk+t  » Vk~t.  We  will 
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show  by  contradiction  that  only  (a)  la  possible.  To  obtain  a contradic- 
tion to  (b)  suppose  f»)  is  true.  Then  the  vector  (V)k+x  - wk“x)  lies  in 
a hyperplane  orthogonal  to  A1  at  each  x along  a trajectory.  In  Eq  (3.21) 
on  the  other  hand,  the  vector  (W*^  - W^“x)  lies  in  a cone  about  the 
vector  B (x)  for  each  x along  the  trajectory.  Inspection  of  Eqs  (3.19) 
and  (3.20)  reveals  that  in  order  for  both  equations  to  be  satisfied  for 
each  x along  a trajectory  the  Inner  product  <B  (x).  A1  (x)>  must  be 
constant  which  Is  Impossible  by  assumption  v.  (Since  B (x)  is  the  time 
derivative  of  most  of  die  state  vector  while  A*  (x)  is  only  the  control 
coefficient  for  the  1th  player,  assumption  v.  Is  a very  reasonable  one 
and  Is  not  likely  to  be  violated  in  any  realistic  problem.)  Thus,  the 
contradiction  to  (b)  Is  establlched. 

The  conditions 


„k+x  . Wk-x  - wk. 


Imply  that  W^x  and  W^t  are  continuous.  Furthermore,  because  of  this 
continuity,  the  HJB  equations  and  thus  the  influence  function  equations 
are  defined  on  the  singular  surface.  The  advantages  of  this  continuity 
were  enumerated  in  the  introduction  to  this  chapter,  and  of  course 
the  validity  of  the  HJB  equations  and  Influence  function  equations  is 
essential  to  obtain  singular  trajectories.  Application  of  the  theorem 
will  be  illustrated  In  the  problems  of  Chapters  IV  and  V and  in  Chapter 
V it  is  shown  that  there  are  problema  for  which  the  theorem  does  not 
hold. 
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IT.  Interceptor-Penetrator  Problems 

In  thin  chapter  we  use  a simple  nodal  t6  Illustrate  the  application 
of  the  theory  presented  In  Chapter  II  and  III  and  also  the  salient 
features  of  the  Intercept  problem  treated  as  a NZSDG.  The  state 
equation  used  to  model  the  vehicles’  notion  is  the  sane  type  as  in  Eq 
(3.1)  of  Chapter  III  so  that  the  control  laws  are  the  "bang-bang" 
variety  with  the  possibility  of  singular  controls.  The  theorem  of  Chapter 
III  which  tests  the  continuity  of  the  Influence  functions  on  the  singular 
surfaces  is  applicable,  and  the  Influence  functions  will  be  shown  to  be 
continuous  on  the  singular  surfaces.  Two  problems  are  considered: 

(1)  e two  player  intercept  problea  with  one  player,  the  attacker  "a", 
atteaptlng  to  reach  a fixed  target  and  the  other  player,  the  defender  "d", 
attempting  to  Intercept  a.  Termination  of  the  game  is  achieved  when  d 
achieves  s separation  distance  from  a of  some  arbitrary  length,  say  l. 

He  will  assume  throughout  thst  termination  always  takes  place  although 
an  important  part  of  any  practical  problem  is  assuring  oneself  that 
termination  can  indeed  be  accomplished.  The  defender's  goal  is  inter- 
ception in  minimum  time  while  the  attacker's  goal  is  minimization  of  his 
final  range  :o  the  target;  (2)  a three  player  problem  which  is  the  same 
as  (1)  except  that  another  defending  player  e is  added.  The  termination 
criteria  is  taken  to  be  when  either  c or  d Intercepts  a.  General  solu- 
tions are  discussed,  and  the  totally  singular  solution  is  solved.^ 

1.  In  considering  the  singular  surfaces  we  will  be  concerned  only  with 
the  singular  surfaces  which  intersect  the  terminal  manifold  since 
these  appear  to  be  the  only  significant  singular  surfaces  in  the 
practical  intercept  problem.  Intermediate  singular  arcs  can  occur  as 
in  Isaacs'  homoddsl  chaueffer  problea  [17],  however  these  intermediate 
arcs  seem  to  occur  when  the  players  sre  initially  inside  each  other's 
turning  radius — -a  case  which  will  not  be  considered  in  this  dissertation. 
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The  theorem  of  Chapter  III  is  shown  to  be  satisfied  so  that  the 
influence  functions  are  continuous  on  the  singular  surfaces.  This 
continuity  allows  the  totally  singular  problems  to  be  solved  numeri- 
cally. Numerical  solutions  of  both  the  two  and  three  player  totally 
singular  problems  are  carried  out  to  illustrate  application. 

Problem  Formulation  - Two  Players 

The  equations  of  motion  for  the  two  players  are 


id  . Vd 


yd  - Vd  sin  Td 
yd  m cd  u 

x*  • Va  cos  ya 
yS  . va  sin  ya 


Define  the  state  vector  x to  be 

xT  - (xd,  yd,  yd,  xa,  ya,  ya)  (4.2) 

Fig.  2 shows  the  inertial  position  and  velocity  of  the  tvo  players 
and  the  inertial  position  of  the  attacker's  target. 

The  termination  criteria  needed  to  end  the  game  is  taken  to  be  the 
satisfaction  of  the  equation 

T lx  (tf),  tf]  - 1/2  [ (xd  - xa) 2 + (yd  - ya)2  - i2]  - 0 (4.3) 

«f 

where  t it  arbitrary  (In  a practical  problem  t could  represent  maximum 
launch  range  for  the  defender's  ordnance.  See  Fig.  4).  The  part  of 
the  Intercept  problem  which  makes  it  a nonzero  sum  game  is  the  difference 
in  the  goals  of  the  two  players.  The  defender’s  gpal  is  minimization 
of  the  game  termination  time  t£  while  the  sttacker'a  goal  is  minimization 
of  the  final  range  to  his  target  (*p,  yT).  The  cost  functions  for  tha 
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Fig.  2.  Inertial  Position  and  Velocity  of  the 
Attacker  and  Defender 
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where  A*  In  Eq  (4.5)  Is  replaced  with  the  partial  derivatives  W*^ 
The  Influence  function  equations  for  the  two  players  are 

V*  ■ ° 

*V° 

!*  tfd  a.  8 . d , ■ dv 

X yd  ■ V xd  *in  y - X yd  C0S  7 ) 


■ 0 


A*y,  - v*  (1%.  sin  y*  - lsy4  cos  y*) 

When  referring  to  the  Influence  function  equations  for  the  attacker  or 
defender,  replace  the  superscript  'V  with  "a"  or  "d"  respectively. 

Note  the  absence  of  either  control  In  the  influence  function  equations 
Eq  (4.7),  (4.8),  and  (4.9)  which  means  that  these  equations  are  coupled 
only  at  the  terminal  manifold  by  the  transversallty  conditions.  The 
transversallty  conditions  are  given  by  Eq  (2.25)  and  In  component  form  are 


/ a u. 

- (x  - x ) 

, a d. 

- (y  - y ) 


x“(tt)  - - 1/  T (tf> 


, a d. 

(x  - x ) 

, a d. 

<y  - y ) 
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*'(tf) 


0 

/ • dm" 

-(x  - X ) 

0 

. a d. 

-(y  - y > 

0 

- ♦a(t.)/»  (t.) 

0 

(x‘  - xT) 

I I 

, a d. 

(ya  - yT) 

, » d. 

(y  - y ) 

0 

0 

- 

- 

where 


t • (x*  - x“)  (V*  cos  Y*  - V“  cos  Y)  + (ya  - yd) 
(V*  sin  y*  - Vd  sin  yd) 


(4.9) 


•nd 


♦*(tf)  - [ (x“  - Xj)  V*  cos  Y*  + (y*  - yT)  V*  sin  Y*]^  (4.10) 


e * 

The  minimizing  controls  u and  v are 


* .d 

u * - sgn  1 


V • - sgn  X 


(A. 11) 


* / * 

except  on  singular  manifolds  where  u * 0 and/or  v • 0 are  employed. 


Appendix  D contains  the  derivation  of  these  singular  controls*  The 

TPBVP  whose  solution  is  equivalent  to  solving  the  NZSDG  posed  consists 

. * * 

of  the  state  equations  Eq  (4.1)  with  the  controls  u and  v substituted 
from  Eq  (4.11)  (or  singular  controls  if  appropriate),  and  the 
influence  function  equations  for  both  players  as  given  by  Eq  (4.7). 
Control  Laws 


The  equations  in  Eq  (4,11)  are  bang-bang  control  laws,  so  that  the 
control  pair  is  always  one  of  the  following: 


a + 

1.  u - - 1 


* + , 
v • - 1 


2.  u 


* + i 

v - - 1 
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* 4-  * 

3.  u - - 1 v 


As  stated  in  Chapter  III,  a closed-loop  control  lav  requires  the 

solution  for  all  the  switching  surfaces.  The  location  of  the  state 

relative  to  these  surfaces  then  completely  specifies  the  nonsingular 
* * 

controls  u and  v Trajectories  lying  along  a singular  surface  have 
controls  from  the  control  pairs  2.,  3.,  or  4.  above.  These  singular 
trajectories  are  discussed  below. 

Singular  Surfaces 

In  Appendix  D the  Legendre-Clebsh  necessary  conditions  are  applied 

* 

to  derive  the  admissible  singular  control  candidates  u - 0 and/or 

* ' a 

v *0.  Additional  requirements  for  a singular  control  u are  that  on 

a singular  trajectory  for  d 

*d  Td  - idYd  - '^d  - • • • “ 0 (4.12) 


1 ^ c°s  t + 1 y<i  sin  y *0  (4.13) 

For  a terminal  singular  trajectory  on  which  u*  is  singular,  using 
Eqa  (4.7),  (4.8),  (4.9)  and  (4.12)  It  can  be  shown  that 

sin  yd  - I - Xdyd  / <XdJd  + *dyd>1/2lt.tf 

CO.  yd  - ( - / (ld2d  + XdJd>1/2  Jt.tf  (4.14) 

tan  rd  - [xdyd  / ■ yd)  1 (*a  ■ xd)'t-t£ 

d 7 d 

By  substituting  the  expressions  for  sin  y and  cos  y from  Eq  (4.14) 
Into  the  Inequality  In  Eq  (4.13)  ws  see  thst  the  Inequality  is  strictly 
sstlsfled. 

Eq  (4,14)  Implies  that  the  flight  path  angle  yd  is  such  that  the 
velocity  vector  for  the  defender  lies  on  his  line  of  sight  to  the 
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attacker  at  termination.  The  situation  la  depicted  in  Fig.  3. 

Similarly,  in  order  for  v*  to  be  singular  Eqa  (4.12)  and  (4.13)  with 

"a"  replacing  "d"  must  hold  along  the  singular  trajectory.  For  a 

* 

terminal  singular  trajectory  on  which  v is  singular 

tan  y*  - ( X*y.  / l*x.  )t.tf  (4.15) 

. [ <y*  - tt>  - (♦*/  *)  (y*  - yd) j 

(X*  - Xj)  - (♦*/  i)  (x“  - xd)  f 

where  ♦*  and  T are  given  by  Eqs  (4.9)  and  (4.10).  Eq  (4.15)  Implies 
that  the  singular  trajectory  for  the  attacker  has  a constant  flight 
path  angle  which  depends'upon  the  relative  position  of  both  players  at 
termination  and  the  target.  Typical  singular  trajectories  are  depicted 
in  Fig.  4. 

Continuity  of  the  Influence  Functions 

The  requirements  of  the  continuity  theorem  in  Chapter  III  are  met 
(by  inspection)  with  the  possible  exception  of  condition  iv.  which  must 
be  checked.  If  condition  lv.  is  met  then  the  Influence  functions  are 
continuous  at  the  junctions  with  and  on  the  singular  surfaces  in  this 
problem.  We  now  show  that  condition  lv.  is  satisfied.  Assume  player 
d switches  from  a nonsingular  to  a singular  control. 

Since  and  W* ^ are  identical  on  an  equilibrium  trajectory, 
j condition  lv.'Eq  (3.14)  applied  to  this  problem  requires  that 

i A*^  - 0 (4.16) 

! along  every  trajectory  in  d's  singular  surface.  Because  A*^j  and 

V* are  identical  on  an  equilibrium  trajectory,  A*"^  -an  be  regarded 
as  the  sensitivity  of  player  a's  cost  function  to  a variation  in  the 
flight  path  angle  yd  of  player  d.  From  Eqs  (3.14),  (4.7)  and  (4.16)  we 
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can  now  develop  a relationship  among  the  Influence  function  variablea  of 
both  player  a and  player  d which  must  hold  on  d'a  singular  surface.  It 
la  then  easy  to  demonstrate  that  the  relatlonahlp  holda  which  In  turn 
Implies  the  continuity  of  the  Influence  functions. 

From  Eq  (3.14)  all  time  derivatives  of  are  also  zero; 
therefore  from  Eq  (4.7)  the  equation  - 0 requires  that 

A*  j sin  yd  - Is  j cos  yd  - 0 (4.17) 


tan  Yd  - 1‘yd  / 

Eq  <4 .171  implies  that  the  flight  path  angle  of  player  d can  be  specified 
in  terms  of  player  a's  influence  functiona.  Since  tan  yd  is  a constant 
on  d'a  terminal  singular  trajectory  and  is  specified  by  Eq  (4.14),  we 
can  write  a relation  bet  een  player  a and  player  d'a  Influence  functions 
v hich  must  hold  on  the  singular  surface. 


AV  > XV  - <»V  ' Xd*d)t-tf 


Since  the  terms  In  Eq  (4.18)  are  constants,  we  need  only  to  assure  our- 
selves that  the  equation  and  thus  condition  lv.,  is  satisfied  at  t^. 

This  verification  can  be  made  from  the  transversal ity  conditions  Eq  (4.8) 

(XV  ' xVt-tf- 1(*‘  - yd) ' - *d)1t-tf  (4a9) 

- tan  yd(tf) 


We  have  nov  shown  that  the  conditions  of  the  theorem  of  Chapter  III 
are  satisfied  for  player  d'a  singular  surface  so  that  the  Influence 
functions  are  continuous  on  this  singular  surface  and  the  HJB  equations 
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And  Influence  function  eijiufloni  ere  defined  on  It*  A symmetrical 
argument  holde  for  player  a'a  singular  surfaces.  It  may  be  poaalble 
that  a player  switches  from  a nonelngular  to  a singular  control  while 
the  other  player  Is  already  using  a singular  control.  This  actuation 
Is  merely  the  Intersection  of  the  two  singular  aurfacea  and  no  unusual 
behavior  Is  associated  with  this  Intersection. 

Totally  Singular  (Long  Range)  Problem 


When  long  ranges  are  Involved  in  the  trajectory  of  each  vehicle 
and  the  turn  radii  of  the  vehicles  are  very  snail  cosipared  to  these 
ranges,  then  the  equlllbrlust  trajectories  are  totally  singular  except 
for  the  initial  turning  segments.  In  the  context  of  the  problea  at 
hand,  the  trajectories  are  straight  lines.  This  sinpllficatlon  nakes 
the  problem's  solution  fairly  simple. 

Suppose  the  trajectory  of  each  vehicle  Is  totally  singular. 

Then  from  Eqs  (4.14)  and  (4.15)  we  have 

tan  yd  - [ (ya  - yd)  / (xa  - xd)]£,t  (4.20] 


a f(y*  - yT)  - (♦*  / T)  (ya  - yd) 


V - Xj)  - (ia  / h (xa  - xd)  t.t 


Using  the  singular  controls  u*  - v*  - 0 and  the  state  equation 
Eq  (4.1)  we  obtain  the  algebraic  equations 

xd(t)  - xd(t  ) + (t  - t ) Vd  cos  Yd 
o o ' 

yd(t)  - yd(t  ) + (t  - t ) Vd  sin  Yd 
o o 

(4 

**(t)  - Xa(t  ) + (t  - t ) V*  cos  v* 

0 0 1 

y*(t)  - y*(to)  + (t  - to)  va  sin  y* 
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let  tg  » 0.  For  convenience  define  a point  (7*,  y**)  along  the  flight 
path  of  the  defender  as 

7*(t)  - x^Ct)  + t cos  Yd 

(A.2 

y^c)  “ yd(t)  + l sin  Yd 
so  that  Eq  (4.22)  becomes 

7*(t)  ■ xdo  + (Vd  t + i)  cos  yd 
3^(0  - yd  + (Vd  t + i)  sin  yd 


x*(t)  - x*  + V*  t cos  Y* 


y*(t)  « y*Q  + V*  t sin  Y* 

At  tf,  the  points  (7*.  7*)  »nd  (**,  y*>  are  identical. 


7*(tf)  • x*(t() 
- y“(t«) 


Eqa  (A. 20),  (A. 21),  (A.  A)  and  (A. 25)  constitute  the  TPBVP.  Note 
that  the  unknowns  ya,  y*,  and  t^  are  all  determined  at  the  unknown 
terminal  point. 

Numerical  Method 


The  following  simple  algorithm  can  be  used  to  numerically  solve 
the  TPBVP  for  /,  y , and  t^.  Let  the  superscript  "o"  Indicate  the 
Initial  guess  while  the  superscript  "1"  indicates  the  computed  value 
based  on  the  Initial  guess. 


\ 


1.  Guess  y , t^  . 

o.o  o o 

2.  Calculate  [x*  (tf),  y“  (tf) ] and  [x*  (t{),  y8  (tf)]  fro«  Eqs 


(4.22),  (4.24)  and  (4.25). 
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Li 


3.  Calculate  V*  from  Eq  (4.21). 

o o 

4.  Calculate  [xa  (t  ),  ya  (t  )]  from  Eq  (4.22)  using  final 

0 o 

conditions  in  2.  and  3.  as  initial  conditions. 

o o 

5.  Determine  the  error  between  [xa  (t  ),  y (t  )]  and 

o o 


[xa(t  ),  ya(t  )]. 


du  o d 

Correct  v“  and  t ° by  setting  y 


tf  +«*• 


d°  . . d . 
Y +oy  and 


d°  o d‘  1 

7.  Replace  y and  t ° by  y and  t^  and  repeat  steps  2.  through 


6.  andA^  may  be  determined  by  a number  of  schemes,  many 

of  which  may  be  implemented  by  using  the  optimization  program 
AESOP  (Automated  Engineering  and  Scientific  Optimization 
Program)  [12,  13].  Continue  the  iterations  until  the  error 
ir  Step  5.  has  been  reduced  to  a suitable  bound. 

The  above  algorithm  is  implemented  to  illustrate  a numerical 


solution. 

Initial  Conditions 

In'tial  Guess 

xd(0)  - 0 

xa(0)  - 2000 

vd 

- 1000 

d° 

Y "1  radian 

yd(0)  - o 

ya(0)  - 1000 

va 

- 500 

tf°  - 

*T-° 

yT  - 500 

1-25 

Program  AESOP  [13]  was  used  with  an  IBM  7094  digital  computer  to 
obtain  the  corrections  Ay*1, 4 t^  in  the  algorithm.  Ay  d and  At^  were 
selected  to  minimize  the  error  in  Step  5. , 

(error)  - (fxa(to)  - xa°(t;o)]2  + Iya(to)  - ya°(to) )2)1/2 

Using  three  minutes  of  machine  time,  the  error  was  reduced  from  the 
Initial  guess  error  of  114  to  a final  error  of  .063.  The  final  error 
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represents  a distance  difference  of  .063  between  the  calculated 
o o 

initial  point  x*  (t  ),  ya  (t  ) and  the  actual  initial  point  xa(t  ), 
o o o 

y*(t0).  The  computed  values  of  Y^ , ya,  and  ^corresponding  to  the 
final  error  are  • 54.6°,  Y*  * 261.6°,  and  » 1.51. 

In  ££  actual  Intercept  problem  the  algorithm  could  be  employed 
to  provide  a sampled  data  feedback  control  law  provided  the  sample 
intervals  are  small.  In  this  control  law  the  state  is  sampled  at 
discreet  times  and  the  sampled  etate  is  used  as  a new  initial  condition. 
Employing  the  algorithm  for  each  state  sample  updates  the  flight  path 
angles  yd  and  y*  and  Che  time  to  intercept,  tf. 

If  one  of  the  players t say  a,  employs  a nonequilibrium  control 
law,  this  feedback  control  law  for  player  d Insures  a better  final 
cost  for  player  d.  It  should  be  noted  that  the  smaller  the  sample 
interval  is  the  quicker  will  be  player  d’s  reaction  to  nonequilibrium 
play  by  player  a. 

Three  Player  Formulation 

Three  player  differential  games  have  been  solved  for  pursuit- 
evasion  problems  [4],  however,  the  games  are  posed  as  zero-sum  games 
with  the  third  player  introduced  by  some  artifice  such  aa  a constraint 
or  a specified  guidance  law  rather  than  as  an  ^.(dependent  minimizing 
player.  In  this  section  an  additional  independent  pl*y«  will  be 
added  to  the  problem  of  two  players  already  presented  in  this  chapter. 
The  selection  of  the  third  player’s  coat  function  dictates  the  degree 
of  his  cooperation  with  the  original  defender  ot  the  two  player  game. 

Consider  the  two  player  problem  in  this  chapter,  and  add  to  the 
state  equation  Eq  (4.1)  the  equations  of  notion  for  the  third  player 
whom  we  shall  call  the  cooperating  player  "c", 


A- 


x-'-V- 


y:  'di 


K 

b , 

e ' 


F * , 
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c „c  . c 
y - V ain  y 


YC  • cC  w | w j < 1 
The  state  vector  is  now 

T.dddcccaaav  t,  „ 

x ■ (x  , y , t , x , y , y , x , y , y ) (4«27 

Suppose  that  the  goal  of  the  cooperating  player  is  identical  to 
the  original  defender's  goal  of  interception  of  the  attacker  in  minimum 


Depending  on  the  relative  positions  of  the  players  and  assuming  termi- 
nation can  take  place,  one  or  the  other  of  the  defenders  achieves 
intercept  first  with  the  possibility  of  simultaneous  intercept.  If 
there  were  a termination  criteria  such  as  Eq  (4.3)  for  each  of  the 
defenders,  the  termination  would  be  ambiguous.  To  avoid  this  problem 
a new  single  termination  criteria  Is  formulated  which  includes  the 
termination  criteria  for  each  player, 

» (*  <t{),  tfl  - 1/2  [ Td]  l ¥r]  (4.29 

. t,  d a.2  , d Sv2  .2, 

- 1/2  [(x  - x ) + (y  - y ) - 1 ] 

x , c a.  2 . c aN2  , 2. 

l(x  - x ) + (y  - y ) - k J 


k Is  the  radius  of  c's  capture  circle  and  i Is  the  capture  circle 
radius  for  d.  The  game  is  terminated  the  first  time  Eq  (4.29)  is  sat- 


TPBVP  for  the  Three  Player  Problem 


The  set  of  Influence  function  equations  for  this  problem  are  the 
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equations  in  Eq  (4.7)  plus  the  equation#  for  the  third  player, 

>%d-° 


»V  ' V<1  <»\d  *ln  yi  - AV  C0*  fd> 


X*  , " VC  (\axC  sin  TC  - A*yC  cos  YC) 


X , * o 

x* 


A8^.  ' V*  (x“xa  Sin  Y“  - XSy,  cos  y*) 
where  a Is  one  of  tne  set  (d,  c,  s)  which  refers  to  the  defender, 
cooperating  defender  and  attacker  respectively,  thus  Eq  (4«30)  contains 
27  component  equations. 

The  transversality  conditions  are 

xsctf)  - u%  - <;s  / i)  rx]t.tf  «•»> 

Eq  (2.25)  Is  expressed  In  component  fora  In  Appendix  E for  this  problem. 

The  bsng-bang  control  lews  for  the  orlginsl  defender  and  attacker 
remain  unchanged  while  the  control  law  for  the  cooperating  defender  has 


the  same  forj, 


sgn  X yC 


The  singular  control  for  each  player  Is  unchanged, 


0 v - 0 w « 0 
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The  necessary  conditions  for  player  d to  have  a singular  control  are 

" 0 (*.3 

X*1^  sin  y*  + cos  y*  4 0 (4,3 

and  if  the  problem  terminates  with  a singular  control  u - 0,  then  on 

the  singular  trajectory 

, d , ,d  , , d2  d2  ,l/2i 

Sin  Y - [ - 1 yd  / (X  xd  * X yd)  lt.tf 

...  f 


tu.  / - (<y*  - yd)  / (x*  - xd)lt.e 


For  player  c to  have  a singular  control  v ■ 0, 

' 0 (4.36) 

*Cxc  *ln  TC  + XCyc  cos  y°  - 0 (4  .37) 

* 

and  if  the  problem  terminates  in  a singular  control  v ■ 0,  then  on  the 


singular  trajectory 


•in  TC  - ( - XCyC  / ♦ Xc2yc)1/2)t.t 


c2  ,1/2, 


cos  Y - l - X"xC  / (X  xC  + X yc>  ' ]t.t 


tan  YC  ■ l(y*  - yc)  / (**  - *e))t.c 


Finally,  for  v to  be  a singular  control 
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X*^  ein  T*  + X*^  cos  y*  - 0 (4.40) 

* 

and  If  the  problem  terminates  In  a singular  control  v - 0,  then  on 
the  singular  trajectory 


tan 


a 


Y 


(r*  - rT>  - (♦*  / *)  it/ ay* 

<**  - xj)  - <♦*  / t)  3»/Jx*  t.t^ 


(4.41) 


The  Inequalities  of  Eqs  (4.34),  (4.37)  and  (4.40)  are  shown  to  hold  with 
strict  inequality  in  the  sane  manner  as  in  Eq  (4.13). 

Because  of  the  termination  criteria  Eq  (4.29),  one  or  the  other  or 
both  of  the  defending  players  will  make  the  intercept  and  cause  termi- 
nation. Recalling  the  termination  criteria  function 


t [x  (tf),  t£l  - 1/2  I yd]  [ yc]  - 0 


(4.29) 


T ■ 0 Implies  the  following  possibilities; 

(•) 

Yd  - 0 

(playmr  d intercepts  first) 

(b) 

»c  - o 

(player  c Intercepts  first) 

(c) 

rd  - yc  - o 

(simultaneous  intercept) 

If  (a)  holds  In  a problem,  then  examination  of  the  transversal Ity 
conditions  In  Appendix  E reveals  that  the  trajectory  of  the  attacker 
does  not  depend  on  the  position  of  the  nonintercepting  player  c.  In 
other  words,  the  attacker  and  Intercepting  player  d have  the  same 
trajectories  as  In  the  two  player  game  where  the  nonintercepting  player 
c Is  excised  from  the  problem.  The  nonintercepting  player's  trajectory 
Is  determined  by  Eq  (4.38)  which  Implies  that  If  a singular  arc  Is 
attained  by  player  c,  at  termination  his  line  of  sight  to  the  attacker 
Is  coincident  with  his  velocity  vector.  The  situation  is  depicted  in 
Fig.  5.  A similar  statement  holds  for  case  (b). 


(the  cooperating  defender)  is  added  to  the  problem  and  if  his  capability 
permits,  the  three  player  problem  solution  will  differ  from  the  solutions 
for  cases  (a)  and  (b)  where  the  attacker  and  one  of  the  defenders  play 
as  if  the  second  defender  were  not  present.  When  the  cooperating 
player  can  Influence  the  solution,  but  not  force  case  (b)  then  case  (c) 
holds  and  the  Intercept  is  made  simultaneously  by  both  defenders* 
Examination  of  the  transversal! ty  conditions  for  the  attacker  in 
Appendix  E,  reveals  that  for  case  (c),  X*^a  (t^)  and  Xa  a(t^)  are 
indeterminate.  Application  of  l'Hospital's  rule  to  Eq  (A. 41)  removes 
the  indeterminacy.  The  resulting  equation  for  tan  y*  is 


tan  y*(t.)  - 


y*  - yT  + i*  [<yc  - y*)  / *c  + (yd  - y*)  / »dl 


:*  - Xj  + ♦*  t(*c  - X*)  / ic  + <xd  - x*)  / rdj 


(4.43) 

t-t. 
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The  expreeelon*  for  ♦*,  end  V1  ere  found  in  Appendix  E. 

The  theorem  of  Chepter  III  la  satisfied  on  the  terminal  singular 
aurfaces  of  all  three  players.  This  can  be  verified  in  the  same 
manner  aa  in  the  earlier  two  player  problem  of  this  chapter. 

tfe  can  now  proceed  to  develop  the  totally  singular  solution. 
Totally  Singular  Three  Player  Problem 


As  in  the  two  player  problem,  if  the  turn  radii  are  assumed  very 
small  compared  to  the  total  range  of  each  player,  then  the  problem 
reduces  to  one  in  which  the  trajectories  are  totally  singular  - in  this 
problem,  straight  lines  in  the  plane.  Following  the  format  of  the  two 
player  totally  singular  problem,  the  equations  defining  the  flight  path 
angles  are  given  by  Eqa  (4.35),  (4.38)  and  (4.41)  repeated  here  for 
reference, 

tan  yd  « I(y*  - yd)  / (xa  - xd>)  (4.35 

' Cf 

tan  Tc  - [(y*  - yc)  / (x®  - xC))t_t  (4.38 

. (y*  - y_)  - ( **  / Y)  3Y/  3ya  I 


(*  - *j)  - ( ♦ / y)  3Y/3xa  t_£ 
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x*(t)  - x*0  + Va  t cos  y* 
ya(t)  - y*o  + V*  c sin  Y* 

Define  x*1,  y^,  x0,  y0  by  the  following  equetions, 

x^(t)  « xd  + ( V**  t + 1)  cos  yd 
0 

y**(t)  * yd  t ( Vd  t + 1)  am  yd 

0 (4.45) 

^(t)  - xC  + ( VC  t + k)  cos  yC 
, 0 

^(t)  - yc0  + ( Vc  t + k)  sin  yC 

When  t -<  tj,  one  of  three  conditions  holds;  either 

(a)  x^tj)  » xa(tf)  and  y*1  Ct£)  * ya(tf) 

or  (b)  x^tj)  « x“(tf)  and  y'ftj)  - y*(tf)  (4.46) 

or  (c)  both  (a)  and  (b). 

Case  (a)  corresponds  to  intercept  by  player  d,  case  (b)  corresponds  to 
Intercept  by  player  c and  case  (c)  corresponds  to  simultaneous  Intercept. 

The  TPBVP  consists  of  finding  the  angles  y*,  yC,  and  yd  which  are 
defined  at  the  unknown  terminal  point  by  Eqs  (4.35),  (4.38)  and  (4.41) 
such  that  the  conditions  in  Eq  (4.46)  are  met. 

Numerical  Method 

An  algorithm  which  can  be  used  to  solve  numerically  for  the  unknown 
angles  in  the  TPBVP  is  based  on  the  three  intercept  conditions  of  Eq  (4.42). 
Algorithm 

1.  Assume  intercept  by  player  c. 

2.  Solve  the  two  player  (c  and  a)  NZSDG  using  the  algorithm  for 
the  two  player  totally  singular  problem.  This  provides  an 
intercept  time  for  player  c. 
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3.  Determine  whether  player  d la  capable  of  intercepting  flrat 
in  the  aolution  in  2.  by  computing  player  d'a  time  to  inter- 
cept. If  ao,  the  assumption  in  1,  ia  not  correct,  go  to  4. 

If  not  ao,  the  aaaumption  la  correct;  determine  player  d'a 
flight  path  angle  by  Eq  (4.32). 

4.  Aatume  intercept  by  player  d. 

5.  Solve  the  two  player  (d  and  a)  NZSDC  uaing  the  algorithm  for 
the  two  player  totally  alngular  problem. 

6.  Determine  whether  player  c la  capable  of  intercepting  firat  in 
the  aolution  in  5.  by  computing  player  c'a  time  to  Intercept. 

If  ao,  the  aaaumption  in  4.  la  not  correct;  go  to  7.  If  not  ao 
the  aaaumption  la  correct;  determine  player  c'a  flight  path 
angle  by  Eq  (4.38). 

7.  Reaching  thia  point  impliea  aimultaneoua  Intercept.  The  TPBVP 
which  muat  be  aolved  conalata  ui  aatiafying  the  initial  conditiona 
for  the  problem,  the  tranavereality  conditiona,  Eqa  (4.35), 

(4.38)  and  (4.41)  and  the  termination  criteria,  Eq  (4.42c). 

Eq  (4.43)  are  the  equatlona  of  motion  for  the  three  vehlclea. 

The  following  steps  > It Ida  a numerical  aolution  to  the  almultaneouf 
Intercept  situation: 

(a)  Guess  y 

Using  Eqa  (4.45)  and  (4.46c),  aolve  for  yC  and  t{°  which  gives 
simultaneous  Intercept  for  players  c and  d. 

(c)  With  rd  . yC  . and  tf°  calculate  [xc  (t£),  yc  (t{)]  and 

<j°  d°  o o 

Is  (tf).  y (tf)J.  Compute  [x*  (tf),  y*  (tf)J  from  Eqa  (4.24) 

and  (4.25). 
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o 

(d)  Using  Eq  (4.38)  solve  for  y*  • 

o co 

(e)  Using  y*  an<*  t^°,  compute  [x*  (t^) , y*  (to)]  from  Eq  (4.44). 

o o 

(f)  Compute  the  distance  error  between  [xa  (t  ),  y (t  )]  end 

o o 

the  initial  point  for  player  a,  [xa(t  ),  y*(t  )J. 

o o 

d^  j 

(g)  Define  y by  the  equation  y » y + Ay 

d°  d1 

(h)  Replace  y in  (a)  by  y . 

(i)  Repeat  steps  (b)  through  (h)  untiJ  the  error  in  (f)  has 
been  reduced  to  some  suitable  bound.  Program  AESOP  [13]  may 
be  U3ed  to  obtain  the  correction  &y^  to  reduce  the  error  in 
(f)  to  nearly  zero.  The  error  is  computed  as  in  the  two 
player  problem. 

The  algorithm  is  implemented  to  illustrate  a numerical  solution. 

The  parameter  values  of  the  two  player  totally  singular  problem  are 
used  so  that  the  effect  of  the  third  player  on  the  trajectories  of  the 
origlncl  two  players  may  be  observed.  Three  different  speeds  for  the 
third  player  c will  be  used  to  illustrate  the  three  cases  in  Eq  (4.42). 


Initial  Conditions 


xd(0) 

- 0 

xa(0)  - 

2000 

Vd  - 1000 

Case 

(A) 

vc  . 

500 

yd(0) 

- 0 

ya(0)  - 

1000 

Va  - 500 

Case 

<B) 

vc  - 

1000 

xc<0) 

- 0 

*T  - 

0 

i - 25 

Case 

(C) 

vc  - 

1500 

yc(0) 

- 1500 

yT  ’ 

500 

k - 25 

(j  c a 

The  computed  values  of  y , y , y and  t ^ for  cases  (A) , (B)  and 
(C)  are  given  in  Table  I. 
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1 

i 


Table  I 


Numerical  Result*  for  the  Three  Player  Problem 


Case 

a 

T 

c 

■y 

d 

T 

*f 

Target 

Range 

(A) 

261.6* 

116.1* 

54.6* 

1.51 

1311 

(B) 

253.6* 

120.1“ 

59.9* 

1.47 

1317 

(C) 

247.6* 

115.3* 

62.1* 

1.09 

1526 

t 

l 

l 


The  trajectories  of  the  three  player  game  for  case*  (A)  (B)  and  (C) 
are  depicted  In  Fig.  6.  Note  that  In  case  (A)  the  trajectories  for  player* 
a and  d are  Identical  to  the  two  player  game  since  the  capability  of  the 
added  player  c Is  not  sufficient  to  cause  the  attacker  to  change  his  flight 
path  angle  y*.  When  player  c has  sufficient  speed  to  affect  the  solution 
but  not  to  effect  Intercept  by  himself,  the  attacker  changes  his  flight 
path  angle  from  that  of  the  two  player  problem  so  as  to  cause  a 
simultaneous  intercept.  Finally  If  c's  speed  Is  sufficiently  large,  the 
attacker  plays  only  against  c as  in  cave  (C). 

Summarising,  in  this  chapter  two  simplified  intercepter  penetrator 


■ problems  were  formulated  using  NZSDG  theory.  The  continuity  theorem  of 

t Chapter  III  was  shown  to  hold  on  the  terminal  singular  surfaces  which 

f Implied  influence  function  continuity  on  the  surfaces.  This  continuity 

r permitted  Che  numerical  solution  of  the  totally  singular  problems. 

S' 
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V.  A Pursuit-Evasion  Problem 

Pursuit-evasion  problems  have  classically  been  posed  as  two-player 
zero  sum  differential  games  in  which  the  goals  of  the  players  are 
always  opposite.  However,  In  real ' tic  situation*  It  is  not  always 
clear  that  the  goals  of  the  player*  are  exactly  opposite  so  that  zero 
sum  differential  games  do  not  suffice  to  model  the  situation.  It  is 
this  aspect  with  which  this  chapter  is  concerned.  Here  we  solve  s 
two  player  fixed  terminal  time  NZSDG  in  which  the  players  have  conflicting 
but  not  diametrically  opposite  goals. 

Denoting  the  "pursuer"  by  p and  the  "evader"  by  e,  the  goal  of  p 
is  to  minimize  a function  of  the  final  relative  range  and  p's  line  of 
sight  error  while  the  goal  of  e is  to  minimize  a different  (but  not 
opposite)  function  of  the  relative  range  and  p's  line  of  sight  error. 
Relative  range  and  the  line  of  sight  error  of  the  pursuer  are  chosen 
because  of  the  importance  of  these  functions  in  pursuit-evasion  problems. 
The  dynamic  model  for  the  two  vehicles  is  taken  from  Ref.  (5]  where  it 
was  used  in  the  analysis  of  a zero  sum  differential  game  of  the  pursuit- 
evasion  type.  The  problem  considered  in  this  chapter  will  be  shown  to 
satisfy  the  theorem  of  Chapter  III  which  implies  continuity  of  the 
influence  functions  when  the  controls  are  discontinuous  as  the  trajectory 
joins  a singular  surface. 

The  objective  of  this  chapter  is  to  :haracterize  the  solutions  to 
a NZSDG  pursuit-evasion  problem;  therefore  solutions  will  be  completed 
only  to  the  extent  necessary  to  illustrate  the  solution  behavior. 
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P rob  lea  Forauleclon 

The  dyraaic  model  for  this  problem  consists  of  two  vehicles  moving 
in  a plane.  The  equations  governing  the  motion  for  each  vehicle  in  a 
vertical  place  are 


x » V cos  7 


y - v sin  Y (5.1) 

V » (T-D)/ra  - g sin  y 
7 - L/m  V 

where  x and  y are  spatial  coordinates  in  the  plane  of  motion,  V is  the 
speed,  7 the  flight  path  inclination  v.r.t*  the  x axis,  T the  thrust,  and 
m the  mass.  The  aerodynamic  forces  are  defined  by  the  drag  and  lift 
equations  D - 1/2  p V2  S CD  and  L - 1/2  p V2  S CL- 

If  the  induced  drag  due  to  lift  is  small  compared  to  the  total  drag, 
it  is  possible  to  approximate  the  drag  D by  assuming  zero  Induced  drag 
due  to  lift.  This  approximation  is  especially  appropriate  when  the 
vehicle  speed  is  great  and  the  acceleration  in  the  lift  vector  direction 
is  limited  because  of  structural  or  pilo*  considerations.  To  show  this, 
define  the  load  factor  n as  the  ratio  of  lift  force  magnitude  to  vehicle 
weight. 


For  a vehicle  with  a load  factor  limit  n the  maximum  lift  coefficient 

max 

C.  is  dependent  upon  the  vehicle  speed  according  to  the  equation 


■ K <11,  g,  p,  S,  n ) 


1/2  pV‘s 
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Assuming  naMlx»  g»  P>  S constant  for  a given  flight  regime, 


is  inversely  proportional  to  the  square  of  the  speed.  Assume  that  the 
total  d/ag  coefficient  can  be  represented  by  the  drag  polar  equation 

9-  (5.4) 


S * S + 

o 

where  C_  is  the  zero  lift  drag  coefficient.  Then  C.  is  aleo  a 


function  of  vehicle  speed 

„2 


+kK' 


T 


(5.5) 


For  a typical  supersonic  fighter  aircraft  a graph  of  Cp  vs.  Mach 

o 

nuober  is  depicted  in  Fig.  7 which  shows  that  CD  approaches  an 

o 

assywptatlc  value  at  speeds  above  M » 2.5.  For  the  same  aircraft  Fig.  8 
ahows  typical  graphs  of  Cj.  and  Cu  vs.  V.  At  V » 2700  ft/sec.  the 
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maximum  induced  drag  factor  kC^  ia  leas  than  52  of  the  zero  lift 

■ax 

drag  coefficient  • Thus  at  high  speeds  the  Induced  drag  due  to 
o 

lift  can  be  neglected  without  significantly  affecting  the  dynastic  model. 

Using  the  load  factor  n as  a control  variable  and  neglecting  the 
Induced  drag  factor  kC^2,  the  equations  of  mot. on  for  a vertical  plane 


x - V cos  y 
y - V sin  y 

V - T/m  - 1/2  pV^S  Cj  - (g  *in  t) 

a o 


where  the  control  variable  n is  constrained  according  to  the  Inequality 


Since  gravity  affects  both  vehicles  almost  equally,  gravitational 
effects  will  be  neglected.  If  desired,  after  the  control  laws  are 
determined  gravity  can  be  replaced  in  the  problem  and  the  trajectories 
computed  to  give  approximate  equilibrium  trajectories  in  the  presence 
of  gravity  [25].'  Considering  two  vehicles,  a pursuer  p and  evader  e 
the  state  equations  are  (neglecting  gravitational  effects  in  the  V equations) 


cos  yk 


yP  - Vp  sin  yp 


VP  - Tp/»p  - D Vp2  Cn  P 
P ** 


YP  - npg/VP 


e ,,e  e 

X * V cos  y 
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ye  - Ve  »in  Y* 

V*  - T*/me  - De  V*2  CD  * 


neg/Ve 


where  D * 1/2  p Sp/np  and  D “ 1/2  p Se/ae 
P * 


The  controls  np  and  n*  are  constrained  according  to  the  inequalities 


<5. 


Fig.  9 shows  the  coordinates  of  the  two  players  as  well  as  the 
"angle-off"  angle  8 and  the  range  R between  the  vehicles.  The  range  R 
and  angle  off  6 are  defined  by  the  equations 

R - i (**  - xp)2  + (ye  - yp)V/2 


(5.11 
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e - ten  -1  (ye  - yp)  - Yp 

xe  - xp 

The  problem  le  to  find  the  equilibrium  controls  np  and  n* 


for  the  terminal  cost  functions 


■Jp  » 4P  [x  (tj),  t£] 


J*  - ♦*  [x  (tf),  tf] 

subject  to  the  state  equations  Eq  (5.8)  and  the  termination  criteria 
constraint 

*[X  (t£),  t£]  - T - t£  - 0 (5.13) 

T It  a fixed  number.  To  make  this  problem  a NZSDC  problem  we  require 
Jp  i - J*  (5.14) 

Necessary  Conditions 

Recall  from  Chapter  II  the  two  different  formulations  of  the 
equilibrium  solution  necessary  conditions  - the  HJB  partial  differential 
equations  Eq  (2.14)  and  the  generalized  Euler-Lagrange  equations  Eq  (2.20). 
For  this  problem  the  HJB  equations  are 


W*t  - - min  H 


where  the  boundary  conditions  for  Eq  (5.15)  are 
H*  - ♦*  (x  (t£),  tf] 

The  Hamiltonian  function,  are 


H*  - X^p  VP  cot  yP  + X*yp  VP  tin  YP  + X*yp  (Tp/mp  - DPVP  Cp  ) + 
l*vp  npg/Vp 
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+ X®  e Ve  co.  Y*  + A®  . V*  .in  y*  + A®y.  (Te/m*  - dV2  CD  P) 

* y o 

+ ^ n®g/Ve  (5.17) 

vhere  W°x  replace.  A®  vhere  H*  i.  u.ed  in  Eq  (5.15). 

The  equillbriuB  control*  np*  «nd  ne*  are  found  from  the  minimization 
in  Eq  (5.15),  to  be 

np  • - np  sgn  Wp  „ (5.18) 

max  ® yV 

n*  “ - ne  *gn  W*  . (5.19) 

max  * ye 

Eq.  (5.18)  and  (5.19)  hold  except  where  WPyP  - 0 and/or  W*ye  - 0 on 
a nonzero  time  Interval.  In  the  latter  caae  the  poaalbllity  of  a singular 
solution  exists  and  oust  be  checked. 

The  HJB  equations  Eq  (5. 15)  are  the  moat  general  fora  of  the 
necessary  conditions;  however,  solution*  to  the  HJB  equations  have  not 
been  found  for  this  problem  so  that  we  are  forced  to  use  the  Bore 
restrictive  Euler-Lagrange  equations  which  for  this  problen  consist  of 
the  state  equation  Eq  (5.8)  and  the  following  Influence  function  equation*: 

A®xp  - 0 

X*  „ - 0 
yP 

A*vp  - ' A®xp  co,.  yp  - A*yP  .in  yP  + 2 X*yp  VP  Dp  C^P  + X*yP  gnP*/Vp2 

•'*  P - A*  D Vp  sin  yP  - A*  Vp  cos  YP  (5.20) 

xP  yP 


xv-° 


- - X*  , coa  Y*  - a*  e Bin  y*  + 2A®ye  V*  Dg  * + X*  , g n*  /V®2 
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A*t.  ■ **xe  V*  sin  r®  - ^*ye  v*  c0»  t* 

where  e ie  repleced  by  either  p or  e. 

The  minimizing  equilibrium  controle  np  and  ne  are  obtained  from 
Eqe  (5.18)  and  (5.19)  where  WpyP  end  W*ye  are  replaced  by  XpyP  and  Xe^e 

reepectlvely  on  an  equilibrium  trajectory, 

nP*  * '•  nPmar;  "8°  (5-21) 

n‘*  ’ ' n*max  *8n  *®ya  (5.22) 

When  Xpyp  and/or  A*  e are  identically  zero  on  a nonzero  time  interval 

the  possibility  of  a singular  solution  must  be  investigated  as  in  Chapter  II. 

The  tsansversality  conditions  for  the  Influence  function  equations 
Eq  (5.20)  are 

xp  (tf)  - *PX  Is  <Cf),  tf} 

(5.23) 

X*  (t£)  - 9®x  [X  (tf),  t{) 

Singular  Controls 

A detailed  study  of  intermediate  singular  arcs  is  beyond  the 
scope  of  this  dissertation  hence  ve  will  consider  only  the  Important 
class  of  singular  surfaces  vhlch  intersect  the  terminal  manifold. 

(In  problems  with  realistic  initial  conditions  the  intermediate  singular 
arcs  are  not  likely  to  occur). 

Eqs  (5*21)  and  (5.22)  indicate  that  the  controls  np*  and  ne*  are  on 
their  respective  constraint  boundaries  except  possibly  on  singular  surfaces 


XPT„  - 0 and/or  X*^  - 0 
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To  characterise  the  control  laws  on  these  singular  surfaces,  the 
necessary  conditions  for  singular  controls  are  employed.  The 
resulting  control  laws  are 

np*  - 0 (5.2! 

for  the  pursuer’s  singular  surface  and 

ne*  ' 0 (5.2< 


for  the  evader's  singular  surface.  Appendix  D contains  the 

analysis  leading  to  Eqs  (5.25)  and  (5.26), 

The  control  sequence  for  the  pursuer  is  comprised  of  control 

segments  from  the  set  ( -n^  0,  +n*>roax)*  Similarly,  the  control 

sequence  for  the  evader  is  comprised  of  segments  from  the  set  (-n6^  , 

°.  +!>*)• 

max 

Influence  Function  Continuity  on  a Singular  Surface 

We  now  show  that  this  problem  satisfies  the  requirements  of  the 
theorem  in  Chapter  III  for  a certain  class  of  cost  functions,  which  implies 
continuity  of  the  influence  functions  on  the  singular  surfaces. 

Conditions  i.  of  the  theorem  is  satisfied  which  can  be  verified  by 
examining  the  state  equation  Eq  (107).  By  letting  U9  be  the  control  and 
defining 


n8  - n8max  Us  s - P>  t (5.27) 

|U8|  < 1 (5.28) 


we  see  that  condition  ii.  is  satisfied. 
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Condition  111.  is  satisfied  since  on  either  side  ot  a singular 
surface,  all  controls  are  constant,  and  we  will  make  the  reasonable 
assumption  that  the  neighborhood  under  consideration  contains  no 
control  switching  surface  except  the  singular  surface.  Condition  v. 
is  satisfied  by  inspection  of  the  state  equation  Eq  (5.8).  Only  condition 
iv.  remains  to  be  shown.  Assume  p has  a singular  control.  Letting  "i" 

In  the  theorem  be  replaced  by  "p"  we  must  have  from  Eq  (3.15) 

KCYp  - 0 (5.29) 

on  p's  surface.  Since  singular  trajectories  proceed  along  the  surface 
itself,  we  will  replace  We^p  with  Xe^p.  For  every  trajectory  in  p's 
singular  surface  we  thus  require 

ieyP  - 0 (5.30) 

The  condition  in  Eq  (5.30)  can  be  met  at  the  terminal  surface  provided 
the  transversal ity  condition  for  Xc^p  (tf)  is  identically  zero 

XeyP  (tf)  - (3*e/3TP)t.t{-  o (5.31) 

The  condition  in  Eq  (5.30)  can  be  maintained  on  trajectories  in  the 
singular  surface  provided 

leYp  (t)  - 0 (5.32) 

on  the  surface.  Eq  (5.20)  and  Eq  (5.32)  imply  that  on  trajectories  in 
p's  terminal  singular  surface 

TfP  - ‘ (tf)/X%P  (tf)  (5’33) 

In  Eq  (5.33)  tan  yP,  which  is  a constant  on  p's  singular  surface,  is 

specified  In  terms  of  e's  Influence  functions.  FroD  the  transversality 

conditions  tan  yP(tf)  * (XP  0AP  „)  so  that  on  p's  terminal  singular 
1 yp  Xp  t-tf 
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surface  the  equality 

*V*\p  * < VXP*P> 

t-tf  {5.34) 

must  hold.  If  Eqs  (S.29)  through  (5.34)  hold  on  p's  singular  surface, 
then  condition  iv.  of  the  theorem  in  Chapter  III  is  met  and  we  conclude 
that  player  e's  Wluence  functions  are  continuous  on  p's  singular 
surface.  A '-vmmetrlc  argument  holds  for  e's  singular  surface. 

Range  and  Angle-Off  Cost  Functions 

A class  of  cost  functions  for  which  Eqs  (5.29)  - (5.34)  holds  is 

«P  - [aR  + tl-a)  e2l  a e [0,  1)  (5.35) 

cf 

J*  - [-bR  - (1-b)  02lt.Cf  b £ [0,  1) 

R and  8 are  defined  in  Eqs  (5.10)  and  (5.11).  To  make  the  problem  a 
NZSDG  problem  it  is  only  necessary  to  choose  a ^ b in  Eq  (5.35).  (If 
a - b the  game  is  equivalent  to  a zero  sum  differential  game.)  The  cost 
functions  in  Eq  (5.35)  are  important  in  the  formulation  of  pursuit-evasion 
problems  since  both  the  terminal  range  and  the  terminal  angle-off  are 
significant  parameters. 

We  will  now  characterize  the  singular  surface  for  the  pursuer,  and 
Eqs  (5.30)  and  (5.32)  will  be  shown  to  hold  for  p*s  singular  surface  which 
Intersects  the  terminal  manifold.  In  order  to  have  a singular  surface 
for  p which  intersects  the  terminal  manifold,  from  Eqs  (5.28)  and  (5.31) 
we  require 

*PTP  (tf)  - 0 (5.36) 
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t.a  r»  (tf)  - <VxP*p)t.t 


■o  that  from  Eqa  (5.35)  and  (5.36) 


»P.  (O  - (2  (1-a)  9 (-1)1. 


which  Implies  (for  a + 1) 


(t£)  - 0 


Eq  (5.39)  laplles  that  the  pursuer  haa  hla  velocity  vector  on  the  line 
of  sight  to  the  evader  at  the  final  time.  This  can  also  be  seen  from 
Eq  (5.37)  which  when  evaluated  becomes 

tan  YP  (t£)  - (<3*P/3yP)/0*P/3xp)]c_^  (5  4 

- t(y*  - yp)/(xe  - *p))t„t 


Eq  (5.40)  Implies  that  at  t£  the  flight  path  angle  yp  corresponds  with 
the  line  of  sight  angle.  Thus,  the  singular  trajectory  for  the  pursuer 
Is  a straight  line  whose  direction  coincides  with  the  pursuer's  velocity 
vector  at  t£.  Fig.  10  depicts  the  situation  In  which  a trajectory  for  the 
pursuer  contains  a terminal  singular  arc. 

Next  we  wish  to  show  that  Eqs  (5.30)  and  (5.32)  hold  for  the  cost 
functions  In  Eq  (S.35),  which  means  that  when  the  pursuer  Is  on  his 
singular  surface  condition  iv.  of  tha  theorem  In  Chapter  III  Is  satisfied 
Implying  continuity  of  e's  Influence  functions.  Eq  (5.30)  for  this 
problem  becomes 

*\p  <V  • f-2  <1-b>  0 (5.41) 
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Fig.  10.  Singular  Terminal  Trajectory  for  the  Pursuer 


and  from  Eq  (5.39)  we  conclude 


*\p  «f>  ’ 0 

To  show  Eq  (5*32)  is  true,  ve  have  fro*  the  transversal ity  conditions 

<x'yp/x*xP>  * t<Fe-rp>/u*-*p>)t„t  ’ tan  *p  (tf5  (5-42> 

t-tf  f 

By  substituting  from  Eq  (5.42)  into  the  equation  for  l*^p  Eq  (5.20)  we 
see  that  X*^p  is  zero  on  p's  singular  surface.  Thus  Eqs  (5.30)  and 
(5.32)  are  satisfied  on  p's  singular  surface  implying  continuity  of  e's 
Influence  functions. 

Similar  arguments  hold  for  the  evaders'  singular  surface.  A 
situation  in  which  only  the  evader  has  a singular  terminal  arc  is  depicted 
in  Fig.  11.  Fig.  12  depicts  a situation  in  which  the  singular  surfaces 
of  both  the  pursuer  and  evader  Intersect  resulting  in  a tail-chase  situation. 
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Fig.  11.  Singular  Terminal  Trajectory  for  the  Evader  > 
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Fig.  12.  Typical  Singular  Trajectory  for  both  Pursuer  and  Evader 
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The  control  laws  In  Eqa  (5.20),  (5,22),  and  (5.23)  are  open-loop 
lava  since  they  require  the  solution  of  a two  point  boundary  value 
problem  to  obtain  them.  If,  however,  the  singular  surface*  and  all  other 
switching  surfaces  in  the  problem  are  known,  then  closed-loop  control 
laws  can  te  implemented  simply  by  Identifying  on  which  side  of  a particular 
switching  surface  the  state  is  located. 

In  the  class  of  problems  considered  in  this  dissertation,  the  Influence 
function  equations  are  identical  for  both  the  open-loop  and  closed-loop 
control  lavs;  therefore,  closed-loop  control  lavs  are  theoretically 
obtainable  by  solving  the  open-loop  problem  at  each  Instant  of  time  along 
a trajectory  using  the  Instantaneous  state  as  a new  intlal  condition  for 
the  solution  of  the  TPBVP.  This  method  results  in  the  "open-loop 
feedback'*  control  law  which  for  the  class  of  problems  in  this  dissertation 
is  the  same  as  the  closed-loop  control  law. 

Coat  Functions  Special  Cases 

For  the  range  and  angle-off  cost  functions  of  Eq  (5.35)  we  have  the 
following  limiting  cases 

(a)  a » 1 Pursuer  considers  only  final  range 

(b)  a - 0 Pursuer  considers  only  angle-off 

(c)  b - 1 Evader  considers  only  final  range 

(d)  b - 0 Evader  considers  only  final  angle-off 

Interesting  caaes  result  from  (a)  and  (d)  and  (b)  and  (c) , and 

serve  to  illustrate  applications  of  the  NZSDG  theory  to  pursuit-evasion 
games. 

Case  I.  (a  ■ 1,  t)  » 0) 

This  case  occurs  when  the  pursuer’s  goal  is  to  minimise  the  relative 
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XV  (tf)  * t 2 6 <*e-*p>/R2]t;.t£ 


*‘vp  (tf)  - 0 


X > «£>  * <2 


XV  <£f>  ' - X\p  (tf> 

Xty'  (tt)  “ " X yP  (t£5 


' V«  *1 


(t.)  - 0 


* ,e  <t£)  * 0 


The  open-loop  equilibrium  controls  for  this  problem  are  given  by  Eqs 
(5.21)  and  (5.22) 


nP‘  ■ - nP=*x  8gn  xPyP  (5-21) 

ne*  - - n8^  sgn  A*^  (5.22) 

except  when  the  arguments  Xp  p and/or  Xe  are  Identically  zero  for 
Y Ye 

a nonzero  time  interval. 

By  examining  these  control  equations,  the  control  laws  in  a region 
near  the  terminal  surface  can  be  characterized.  Examining  the  pursuer's 

nA 

control  first,  from  Eqs  15.21)  and  (5.44)  we  find  that  at  t^,  nK  is 
undefined  since  Xp^p  (tf)  *0.  We  thus  require  the  derivative  Xp^p  (t^) 

nA  a.  _ 

to  determine  (tf),  where  t^  is  the  time  an  Instant  before  reaching 
the  terminal  surface, 


nP  (tt)  ’ nP»ax  8gn  IXPYP  <t£)  1 
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" nP««  •*“  vP  •!“  YP  - XPyP  Vp  co.  YP)t.t{ 

■ nPMx  *gn  [ gin  e (t£)J 

Hq  (5.45)  implies  that  the  pursuer',  control  in  a neighborhood  of  the 
terminal  .urf.ce  1.  determined  by  the  line  of  »ight  angle  6.  At  tf 
che  pursuer  employs  his  control  so  as  to  rotate  hia  velocity  vector 
toward  the  line  of  sight.  We  now  show  that  this  situation  holds  fcr  a 
larger  region  and  not  just  at  t£. 

in  Eq  (5.45)  lPyP  can  be  expressed  as 

*PTP  “ vP  *ln  <YPf  + af  “ YP)  (5.46) 

where 

YPf  5 Y p (t£)  0f  = 6 (tf) 

Since  lpyP  (c£)  - 0,  XPyP  is  opposite  in  sign  to  Xp^p  (t£)  for  a period 

of  time  before  reaching  the  terminal  surface.  If  Vp  remains  nearly 
constant  XPyp  is  nearly  periodic,  hence  this  period  of  time  [tj-tj]  is 
determined  approximately  by  setting 

/‘f  iPyp  dt  - 0 (5  47) 

C1 

and  solving  for  t^.  Fig.  13  illustrates  typical  oehavlor  of  Xp^p  vs. 
in  the  Interval  (t^,  t^]. 

A similar  analysis  for  the  evaders  control  ne*  results  in  equattons 
analogous  to  Eqs  (5.45)  and  (5.46), 

n6  * "'nax  38n  U6£Ve£  cos  (yp£  + 9.  - ye£))  (S  48) 

V 

Eq  (5.48)  implies  that  the  evader's  control  in  a neighborhood  of  the 
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Fig.  13.  Typical  Behavior  of  Xp^p  vs.;  yP 


terminal  surface  is  determined  by  the  line  of  sight  angle  9 and 
the  flight  path  angles  of  both  playeis  yP  and  y2.  The  evader  employs 
his  control  law  so  as  to  rotate  the  line  of  sight  away  from  the  velocity 
vector  of  the  pursuer.  We  now  show  that  this  situation  also  holds  in  a 
larger  region. 


X can  be  expressed  as 


2 6,  ve  cos  (Yp.  + 0.  - ye) 


( 

i*  ; 


Since  (tj)  • 0,  ke^c  is  opposite  In  sign  from  le^e  (t^)  for  a 

period  of  time  before  reaching  the  terminal  surface.  As  In  Eq  (5.47)  this 
period  of  time  [t^,  tf]  Is  determined  approximately,  provided  Ve  is  nearly 
constant,  by  Integrating 
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/Cf  co.  (YPf  + 9f  - y*)  dt  - 0 (5.50) 

C2 

and  solving  for 

Fig.  14  illustrates  typical  behavior  of  vs.  y*  backwards 
from  the  terminal  surface.  As  long  as  le^e  has  the  same  sign,  Che  control 

e* 

n is  constant.  The  trajectories  for  p and  e associated  with  Pigs.  13 
and  14  are  shovn  in  Fig.  15.  For  this  problen  the  singular  solutions 
appear  to  be  pathological  since  the  evader's  cost  function  is  defined 
so  as  to  avoid  the  tail  chase  situation  or  the  pursuer  singular  arc 
where  6 (t^)  is  zero.  Therefore,  ve  may  conclude  that  the  important 
control  laws  in  this  special  case  are  those  closed-loop  laws  where  the 
pursuer  employs  his  control  to  force  the  angle-off  angle  6 to  zero  and 
the  evader  employs  his  control  to  force  the  angle  6 away  from  zero 


Fig.  14.  Typical  Behavior  of  Ae^c  va*  Y* 
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Fig.  15.  Typical  Trajectories  for  the  Special  Case 
a " 1,  b * 0 

Case  II.  (a  - 0,  b - 1) 

In  this  case  the  pursuer  considers  only  the  final  angle-off  angle 
8 (tj)  and  the  evader  considers  only  the  final  range  R (tf) . The  cost 
functions  Eq  (5.35)  become 

Jp  - 82  <tf)  (5  51) 

J*  --R  (tf) 

Again  we  will  examine  the  backward  trajectories  to  deduce  the  behavior 
of  the  solutions. 

Transveraallty  conditions  yield  the  terminal  values  for  the  Influence 
functions 

*PXP  (cf)  -(28  <yV>/R2Jt.tf 
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XPyp  <t£>  - [ -2  8(**-xP)/*2]t 


AP  p (t.)  —2  6 (t.) 


xPx«  tV  ‘ - xP*P  <V 

*Pye  (tf>  - * >Pyp  <tf) 


»P„C  <0  - 0 


Xp  e (t.)  - 0 


X<xP  (t£)  ' l<1‘<!-*P>/R]t.tf 
**p  <«f)  - [Cye-yp)/R)t.t 


xt-  <0  - 0 


x\p  Ctf)  - o 

X*x*  (V  ■ ' X*xP  (tf> 


X y«  (V  " x y? 


»V  <v  - ° 


* %.  CO  - 0 


The  open  loop  equilibrium  controls  for  this  proolem  are  the  snae  as 
Eqe  (5.21)  ana  (5.22,  except  for  the  possibility  of  singular  controls. 


Frcn  the  requirement*  for  a pursuer  terainsl  singular  trajectory 
n?*  - 0 


73 


DS/MC/71-3 


I 

I 


I 


i 

t 

s 


and 

Xpyp  - 0 (5.54) 

which  lo  turn  Implies  from  Eq  (S.52)  that 

0 (tf)  - 0 (5.55) 

Eq  (5.55)  Implies  thst  the  pursuers'  velocity  vector  Is  on  the  line  of 
sight  to  the  evader  at  tj.  The  evaders'  singular  arc  la  characterised  by 

ne*  - 0 (5.56) 

and 

l*re  “ 0 <5-57) 

We  have  already  shown  that  the  evader's  velocity  vector  must  also 
lie  on  the  line  of  sight  between  pursuer  and  evader  at  in  order  for  e 
to  be  on  a terminal  singular  arc.  We  therefore  conclude  that  the  singular 
trajectories  for  this  special  case  havi  the  saror  general  character  as 
those  in  Fig3.  10,  11,  and  12. 

Two  special  cases  of  the  cost  functions  in  Eq  (5.35)  have  been 
examined.  In  Case  I where  the  pursuer  considers  final  range  only  and  the 
evader  considers  final  angle-off  only  it  is  found  that  the  singular  surfaces 
play  an  insignificant  role  in  the  problem  solution  due  to  the  fact  tnat 
the  evader's  goal  avoids  the  situation  where  the  final  angle-off  becomes 
zero.  On  the  other  hand,  in  Case  II  where  the  oursuer  considers  only 
final  angle-off  and  the  evader  considers  only  final  range,  the  singular 
surfaces  appear  and  do  have  a significant  role  in  the  solutions. 

The  problem  of  this  chapter  illustrates  that  NZSDG  theory  can 
be  us*d  to  model  a pursuit-evasion  combat  situation  in  which  the  goals 
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of  the  players  ere  not  poaed  aa  being  diametrically  opposite  as  in 
aero  sum  games.  The  solutions  arc  optimal  for  each  player  in  the 
•etwe  of  equilibrium  optimality. 
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VI.  Conclusions 

In  this  dissertation  the  theory  of  Nonzero  Sum  Differential  Games 
is  extended  so  that  the  theory  can  be  applied  to  some  combat  problems 
between  two  or  more  combatants.  Useful  transversality  conditions  are 
derived  in  Appendix  C to  augment  the  existing  theory  presented  in 
Chapter  II. 

A major  difficulty  in  solving  NZSDG  problems  is  chat  in  the  equili- 
brium solution  the  partial  derivatives  of  the  value  functions  and  the 
influence  functions  can  be  discontinuous  on  switching  surfaces  where  the 
controls  are  discontinuous.  However,  for  a certain  class  of  NZSDG  in 
which  the  state  equation  is  nonlinear  with  the  controls  bounded  and 
appearing  linearly,  the  partial  derivatives  of  the  value  functions,  and 
the  influence  functions  are  continuous  across  the  switching  surfaces. 

In  Chapter  III  this  class  of  problems  is  presented  and  a theorem  to 
test  the  continuity  of  the  influence  functions  on  singular  surfaces  is 
derived.  This  is  a significant  result  since  the  continuity  of  the 
influence  functions  makes  the  equilibrium  solution  much  easier  to  obtain. 

Chapter  IV  presents  two  intercept  problems  in  a plane  between  first 
two  then  three  combatants.  The  problems  are  posed  as  NZSDG' s in  which 
the  players  are  first  one  then  two  defenders  whose  goal  is  to  intercept 
the  attacking  player  in  minimum  time.  The  attacking  player's  goal  is 
minimization  of  his  range  to  a fixed  position  target  before  intercept 
occurs.  The  problem  is  shown  to  satisfy  the  influence  function  continuity 
theorem  of  Chapter  III.  The  solutions  are  characterized  by  each  player 
making  a hard  turn  to  a particular  heading  then  making  a straight  dash 
on  that  heading  until  intercept  occurs.  When  the  initial  turns  are 
neglected,  totally  singular  solutions  result  which  can  be  easily  obtained 
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numerically.  An  algorithm  Is  preaanted  which  aolvaa  the  totally 
aingular  problems. 

Chapter  V presents  a pursuit-evasion  problem  of  two  aircraft  in 
which  the  goals  oi  the  players  are  not  diametrically  opposed.  The 
coat  functions  of  each  player  are  different  (but  not  opposite)  functions 
of  the  final  range  and  angle-off.  This  problem  is  also  shown  to 
satlofy  the  influence  function  continuity  theorem  of  Chapter  HI.  The 
solutions  to  this  problem  ara  characterized  by  trajectories  which 
consist  of  hard  turns  and  straight  dashes.  Two  special  cases  are 
discussed  in  which  first  the  pursuer  considers  only  final  range  and 
the  evader  considers  only  final  angle-off.  In  this  case  the  singular 
arc  (straight  dash)  appears  to  have  no  significant  role  in  solutions 
due  to  the  nature  of  the  evader's  cost  function.  In  the  second  special 
case  the  pursuer  considers  only  final  angle-off  while  the  evader  con- 
siders only  final  range.  The  usual  hard  turns  and  straight  dashes  for 
both  players  appear  in  solutions  for  this  case. 

The  results  of  this  dissertation  are  significant  because  the  use 
of  a NZSDC  formulation  to  model  combat  situations  results  in  a more 
general  problem  in  wnlch  the  goals  of  the  players  are  not  required  to  be 
diametrically  opposite  as  in  zero  sum  differential  games.  This  means 
that  conflict  situations  can  be  modeled  with  more  flexibility  and  realism 
to  reflect  the  actual  goals  of  the  opponents. 

Many  problems  involving  the  complex  aircraft  in  today’s  Air  Force 
inventory  can  be  adequately  modeled  —>lng  the  class  of  problems  presented 
in  Chapter  III  and  investigated  in  this  dissertation.  The  state  equation 
of  Chapter  III,  although  linear  in  the  control  variables,  often  serves 
as  a close  approximation  to  the  state  equation  with  nonlinear  control 
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variables,  and  yields  solutions  which  are  close  approximations  of  the 
nonlinear  solution.  For  this  reason, the  combat  models  which  fit  the 
class  of  problems  investigated  give  a basic  insight  into  the  nature  of 
solutions  involving  the  more  complex  models  with  nonlinear  control 
variables  in  their  state  equation. 

The  author  intends  to  Investigate  the  extent  to  which  the  models 
of  aerodynamic  vehicles  can  be  linearized  in  the  control  variables 
so  that  results  of  this  dissertation  may  be  applied.  In  addition,  it 
is  the  author' 8 intent  to  investigate  the  possibility  of  extending 
the  class  of  problems  studied  in  this  dissertation  to  include  the  fully 
nonlinear  models  of  aerodynamic  vehicles. 
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Appendix  A 


Coat  Functions 


The  selection  of  meaningful  cost  functions  is  one  of  the  most 
important  details  in  formulating  any  differential  game,  for  these  cost 
functions  influence  the  control  strategies,  the  trajectory,  and  the 
outcome  of  the  game.  Quantities  to  be  weighted  in  a cost  function  are 
generally  either  point  functions  such  as  terminal  miss  distance  or  path 
functions  which  require  accummulation  by  integration  along  a trajectory. 
The  following  sections  describe  some  meaningful  cost  functions  for  a 


Terminal  Miss  Distance 

If  player  i's  goal  is  to  minimize  the  difference  between  his  state 
(or  components  thereof)  and  player  j *s  state  at  the  termination  of  the 
game,  his  choice  of  cost  functions  can  be 

J1  - ||  x^tf)  - xj(tf)  ||Qi  (A.l) 

where  x*  and  x^  are  the  state  components  of  the  1th  and  j1"*1  players 
respectively;  $ is  a positive  semi-definite  weighting  matrix  used  to 
weight  the  state  components  of  interest. 

Minimum  Time 

If  the  goal  of  plaver  i is  to  minimize  the  time  to  game  termination 
his  choice  of  cost  functions  is  either 


tf 

J1  - / dt 


w 
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where  tf  is  the  smallest  time  such  that  the  termination  criteria  is 


Minimum  Control  Effort  (minimum  fuel) 


Player  i's  goal  of  game  termination  with  the  least  control  effort 
is  reflected  in  the  integral  cost  function 


J - / II  U1  | |pl(t)  dt  (A.4) 

where  P^pi#a  positive  semi-definite  weighting  matrix  function. 

Minimum  Energy 

Player  i’s  goal  of  game  termination  with  the  least  expenditure  of 
energy  is  reflected  in  the  cost  function 

ji  - /f  II  u1  ||2pi  (t)  dt  ca.5> 

co  E 

where  P*E  is  a positive  semi-definite  weighting  matrix  function.  It 
should  be  emphasized  that  caution  and  judgment  must  be  used  whenever  a 
combination  of  various  goals  are  included  in  the  same  cost  function. 

The  goal  of  minimizing  time  as  in  Eq  (A. 2)  and  the  goal  of  minimizing 
fuel  or  energy  as  in  Eqs.-  (A.4)  and  (A. 5)  are  directly  conflicting,  for 
generally,  minimizing  time  requires  maximum  effort.  An  example  of  this 
situation  is  the  cost  function 


J1  - a tf  + (1-a)  r 1 1 U1  | Li  dt 
4 *-o  P plw 


where  0<  a < 1. 


Since  both  goals  cannot  be  met  simultaneously,  the  choice  of  the 
weighting  parameter  a becomes  a matter  of  judgment  based  on  knowledge 
of  the  problem. 


I 


HJB  Equations. 

This  appendix  presents  the  author’s  formal  derivation  of  Influence 
function  equations  Eq  (2.20)  from  the  generalized  HJB  equations 
Eq  (2.18).  The  derivation  is  a generalization  of  Dreyfus’s  work  in 
Ref  (10).  Assume  a normal  problem.  The  HJB  equations  written  in  the 
form  of  Eq  (2.14)  are 

W1,;  - - minfVx  f <x,  t,  U)  + L1  (x,  t,  I))]  i-1 N 

U1 

(B.i; 

where 

U - (U1,  . . . , UN)  (B.2; 

Assume  U1  is  bounded  according  to  the  equation 

K (U1)  < 0 (B.3; 

If  the  partial  derivatives  W*t  and  W*-x  are  known  for  a particular 
point  (x,  t),  then  theoretically  the  equilibrium  controls  U*  can  be 
determined  from  Eq  (B.I)  in  terms  of  x and  t.  If  the  equations  for 
Wix  and  W*t  are  known,  then  equilibrium  trajectories  can  be  con- 
structed forward  in  time  from  the  point  (x,  t),  the  controls  being 
determined  from  the  known  values  of  W*x,  W*t,  x and  t in  Eq  (B.I). 
Proceeding  formally,  the  equation  for  W*x  will  be  obtained.  W*x  can 
be  expressed  as 


w1*  - W + W* 


i =■  1 


N 


(B.4: 


DS/MC/71-1 


I 


! 

Assume  that  Che  equilibrium  control  vector  D*  is  known  in  the  form  i 

5 

B*  - U*  (*,  t)  (B.5)  | 

Then  Eq  (B.l)  can  be  written  j 

W1,  - -W*  f (*,  t,  U*)  - L1  (x,  t,  B*)  i - 1,  ....  N (B.6) 

! 

Taking  a partial  derivative  v.r.t.  x of  both  sides  in  Eq  (B.6)  yields 
W*tx  * ^xx  * - W*x  (fx  + fu*  U*x)  - ft1,  + L1U*  U*x)  (B«7) 

Assuming  W*xt  - W*tx  and  substituting  for  W*xt  into  Eq  (B.4)  from 

Eq  (B.7)  results  in  the  equation  j 

“Sc  ■ - «Sc  £x  + LSe>  -(W1*  f B*  + l1U»  ) B*  i - 1 N 

* (B.8) 

\ 

or 

y 

»lx  - - fx  + L1*)  - ^ fBj*  + L1  Bj*>  If1*  (B-9) 

i - 1,  .....  N 

Suppore  that  U*  is  interior  to  its  constraint  region  defined  in  Eq 
(B.3);  then  the  minimizing  U*  in  Eq  (3.1)  satisfies  the  equation 

WAX  fyi*  + L1  yi*  - 0 (B.10) 

Substituting  from  Eq  (B.10)  into  Eq  (B.9)  gives  the  equation  for  W*x 

. N 

»Sc  ■ - (“Sc  fx  + ^x)  - r (W*  fBJ*  + lV*)  (B.  11) 

J"1  * x 

i - 1 N 
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Defining 

H1  - L1  + W1*  f (B.12) 

and  on  an  equilibrium  trajectory  defining 

A1  : W1,  (B.13) 

Eq  (B. 11)  becomes  Eq  (2.20)  of  Chapter  II,  the  influence  function 
. N 

equations,  X1  - - Hix  - H1^*  uJ*  i » 1 N (B.14) 

Suppose  now  that  U*  is  on  the  boundary  of  its  constraint  region  defined 
in  Eq  (B.3),  then  it  can  be  argued  as  in  Ref  (10)  that  U*x  ■ 0. 
Substituting  U*x  « 0 into  Eq  (B.9)  and  noting  that  the  other  controls 
may  not  be  on  their  constraint  region  boundaries  so  that  rfx(j  4 i) 
is  not  necessarily  zero,  Eqs  (B.ll)  and  (B.14)  are  again  obtained. 

Thus,  the  influence  function  differential  equation  holds  whether  or 
not  U**  is  interior  to  its  constraint  region. 

The  same  general  procedure  as  above  may  be  used  to  show  that  on  an 
equilibrium  trajectorv  the  equation  for  W*t  is 

W1 1 - -H^t  - H1*!)*  D*t  (B.15) 


If  the  state  equation  Eq  (2.1)  is  autonomous  then  Eq  (B.15)  reduces  to 


W*t 


0 


1 - 1,  ....  N (B.16) 
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Appendix  C 

Influence  Function  Transversallty  Condition; 

Three  generally  used  forma  of  the  tranavereallty  conditions  are 
presented  In  this  appendix:  the  Dreyfus  form  [10],  the  Isaacs  form  [17], 
and  the  Berkovltz  form  [5].  The  generalization  of  the  first  two  forms 
to  a NZSSG  Is  due  to  the  author  while  the  third  form  Is  due  to  Sanaa 
[31].,  Since  all  the  forms  are  In  common  use  it  was  felt  that  each 
fora  should  be  presented,  and  the  use  of  any  one  form  is  a matter  of 
preference  and  convenience.  Throughout  the  dissertation,  however,  only 
the  Dreyfus  form  Is  used. 

For  each  form  we  assume  an  n-dimensional  terminal  manifold  In  the 
space  of  x and  t.  For  the  Dreyfus  and  Isaacs  form  the  terminal  manifold 
is  described  by  the  scalar  equation 

1 [ x <t{),  t£]  - 0 (2.3) 

In  Eq  (2.3)  t£  is  the  smallest  time  t such  that  the  equation  Is  satis- 
fied. If  the  final  time  is  fixed,  then  Eq  (2.3)  is  usually  written 
t • T - tf  - 0 

where  T is  a fixed  number. 

Occasionally  Eq  (2.3)  does  not  describe  the  terminal  conditions; 
for  example,  the  terminal  condition  may  be  the  state  reaching  a single 
point  (e.g.,  the  origin).  In  this  case  a n-dimenslcnal  sphere  of 
radius  8 with  the  point  at  the  center  may  be  employed  as  an  n-dimen- 
slonal  terminal  surface  then  a limit  taken  ( * ■*■  0 ) to  determine 
the  transversallty  conditions  [27J.  In  the  case  of  more  than  one 
scalar  equation  describing  the  terminal  surface  it  is  generally  necessary 
to  employ  a Lagrange  Multiplier  technique  such  as  found  in  reference  [10] . 
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Dreyfus  Form 


This  form  assumes  a NZSDC  with  terminal  cost  functions,  l.e. 

L*  ■ 0 in  Eq  (2.4);  so  we  have 

J1  - /ixttj),  t£J  (C.l) 

This  assumption  is  not  overly  restrictive  since  every  problem  with  a 
cost  function  containing  an  Integral  can  be  easily  converted  to  an 
equivalent  terminal  cost  problem  [17].  To  construct  the  equivalent 
terminal  cost  problem  assume  only  the  i^  player  has  an  integral  cost 
function;  define  another  state  component  x^^  such  that  x ■ ( x,  x^^  ^ 


Xn+1  ■ L W*o>  * 0 (C-2) 

Then  the  cost  function  in  the  equivalent  problem  is 

j1  - ♦*[  X (tf),  tf]  + *n+1(t£)  * x (tf),  t£]  (C.  3) 

We  now  proceed  with  the  derivation  of  the  Dreyfus  form  of  the  transver- 
sality  conditions. 

Suppose  the  state  equation  is  written 

x « f ( x,  t,  U*)  x(to>  * x0  (C.4) 

Consider  a point  x(t)  * x^,  t » t^,  such  that  (x^,  t^)  lie  on  the 
terminal  manifold;  that  is 

¥ (xf,  tf)  - 0 (C.5) 

Now  consider  a variation,  5x^  on  the  terminal  manifold.  The  change  in 
the  i6*1  cost  is 

w1  - **  6*f  + *i(tf)  At.  (C.6) 


where  4tf  ia  the  variation  in  terminal  time  induced  by  the  variation 
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fe£.  The  natation  AE£  is  used  to  distinguish  the  1^^'tcdest  Vkt  ie I'ior. 
dx£  from  the  Induced  varlstion  it£ . , is  the  total  tlee  Avrlvatii. 

of  d*.  Similarly, 


dT  - fx  <Sxf  + t (t£)  4tf 

In  order  for  the  equation 

t ( x£  + x£,  t£  + t£)  ■ 

to  hold  to  first  order, 
dt  - 0 


Solving  formally  in  (C.7)  for  dt£  we  have  (assuming  y~^  0 ) 

dtf  - - ;-l(tf)  rXf  d*£ 


(C.7) 


(C.8) 


(C.9) 


(C.10) 


The  Influence  function  X (t)  can  be  regarded  as  the  sensitivity  coeffi- 
cient of  the  cost  J1  to  a change  in  the  state  x at  time  t [10],  so 
that  at  t. 


X1  (t£)  - dJ1  / dx£ 


(C.ll) 


So,  from  Eqs  (C.6)  and  (C.ll)  the  equation  for  the  influence  function 
variables  at  the  terminal  time  is 


x1  ct£)  - [ - ( i1  / ; >rx  ] 


1 - 1,  . a N (C.l?) 


This  equation  may  also  be  written  in  the  form 

X1  <«f>  - t ♦*,  - ( f + ) / ( »*  * + *t  > *x  >«f  (ca3> 

Since  f [ x,  t,  U*  (x,  {A^}f  t)  ] depends  on  the  set  {a^}  generally 

in  a nonlinear  way,  the  equations  in  Eqs  (C.12)  and  (C.13)  are  a set 

of  N coupled  nonlinear  vector  equations  in  the  N unknown  vectors  a*  (t^) . 
Isaacs  Form 

This  form  of  the  transversallty  conditions  is  a direct  extension 
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of  Che  zero  sun  differential  game  tranaversallty  conditions  in  reference 
[17].  There  are  no  reatrlctlona  on  Che  fora  of  Che  cost  functions  In 
this  sethod  so  that  cost  functions  of  the  fora  In  Eq  (2.4)  are  assumed, 

J1  - d*  [x(t£),  C£]  + / L*  [x(t) , t,  U)  dt  1-1 N (2.4) 


The  terminal  manifold  is  described  by  Eq  (2*3).  Since  many  readers 
are  familiar  with  Isaacs*  notation  we  will  borrow  notation  from  Ref. 


Let  the  terminal  manifold  be  parameterized  by  the  equations 
" b^(s^f  82*  • • •»  s^)  i * 1»  . • . • n+1 


where  we  have  let 


s - ( s^  s2, 


h - <hl,  h2 ha+1) 

x - (x,  t) 

On  the  terminal  surface,  the  value  V1  of  the  1th  player  equals  the 
terminal  portion  of  the  1th  cost  function, 

Wi(tf)  - di[x(t£),  t£J  - d1(s)  (C.l 

If  vector  derivatives  of  both  sides  in  Eq  (C.16)  are  taken  w.r.t.  s we 


W1-  x - d1 
x a a 


or  equivalently,  since  xg  - hg  an  the  terminal  surface 


- W1-  hg 


i - 1 H 


In  component  notation,  Eq  (C.18)  can  be  written 
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i 1 

« Z W h, 

*k  3-1  X3  V 


k - 1 (C.19  ) 

1-1 N 


Eq  (C.19)  reduces  to  Isaacs'  form  (17]  If  N Is  set  equal  to  1,  and  f 

and  L*  are  autonomous  (making  W*  *■  W*  * 0 ). 

C n+l 

Eq  (C.19)  provide©  nN  equations  in  the  (n+l)u  unknowns  W*  (tf), 

X3 

i - 1,  • . . , N;  J ■ 1,  . . 4,  n+1.  N additional  equations  are  required 
to  solve  the  system  of  equations,  and  fortunately,  they  are  available. 


From  the  HJB  equations  Eq  (2.18)  we  have 


(W1  £ + L1) 
x 


1-1 N (2.18) 


where  the  equilibrium  controls  have  been  substituted  Into  £ and  L". 

To  cast  Eq  (2.18)  Into  the  notation  o£  this  Appendix,  note  that  W1^  In 
Eq  (2.18)  is  Identical  to  W* 


w1  = w1 

t X_ 


Let  f be  the  augmented  vector 


Then  Eq  (2.18)  can  be  written 


W1-  f + L1  - 0 

X 


i - 1 N 


Evaluating  Eq  (C.22)  on  the  terminal  manifold  by  substituting  the 

parameters  (s^,  s^,  . . .,  s^)  provides  the  M additional  equations 

necessary  to  solve  for  the  w*  (tf).  Eqs  (C.19)  and  (C.22)  are  a set 
*1  < 

of  (rrt-l)N  nonlinear  equations  in  the  (rrt-l)N  variables  W (tf).  Recalling 

1 

11  J 
the  equivalence  of  X and  W ^ on  an  equilibrium  trajectory,  we  see  that 

the  solution  of  Eqs  (C.18)  and  (C.22)  gives  us  the  valuas  of  Xi(rf)  and 


$■>% 


K ■ • 

i 

, i 

1 ■- 
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also  Che  values  of  W (t^).  Caution  must  be  used  In  the  solution 
since  multiple  solutions  can  occur  corresponding  to  termination  on 
different  sides  of  the  terminal  manifold.  In  such  cases  the  physical 
situation  dictates  which  solution  Is  to  be  used. 

Sarma  - Berlcovltz  Form 

A form  of  the  transversallty  conditions  which  closely  resembles 
Isaacs'  has  been  obtained  by  Sarma  [31]  by  generalizing  the  zero  sum 
differential  game  results  of  Berkovltz  [6].  A few  notatlonal  changes 
are  required  to  present  the  results.  In  addition,  Sarma  has  a more 
general  form  of  terminal  surface  consisting  of  the  union  of  a finite 
number  of  n-dimensional  class  C*  manifolds.  The  terminal  surface  T 
la  defined  by  the  equation 


rv 


where  each  le  an  n-dlnensional  C*  aurface.  Each  Is  parameterized 
by  the  equations 

t - T (o) 

3 (C.24 

Xf  " XJ 


where  the  parameter  o la 

« - («j.  <>r  • • • . <>„) 

The  transversallty  equations  are 

a^/ao  + h1  3T^/ao  - x1  ax^/ao  =»  o 

where  the  Index  j refers  to  ths  specific  on  which  Eq  (C.26)  Is 
evaluated.  Eq  (C.26)  together  with  the  HJB  equations, 
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Hit  - - (W1,  f + L1) 


i ■ 1 N 


(2.18) 


evaluated  on  the  terminal  surface  are  sufficient  to  solve  for  the 
unknown  A*  and  W*  on  the  terminal  surface*  (Again  recall  the 
equivalence  of  X*  and  W*x  on  an  equilibrium  trajectory) 


t 


i 
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Appendix  D 

Control  Lava  on  Universal  Surfaces 
In  this  appendix  the  control  lavs  on  singular  surfaces  are  derived 
for  the  problem  in  Chapter  V.  Similar  results  hold  for  the  problem 
of  Chapter  IV.  The  derivation  is  based  on  the  necessary  condition 
for  singular  controls  found  in  Eqs  (2.28)  - (2.30)  repeated  here  for 
reference, 


nVj  * 0 


HV  u1 


H^l  -*Hiui  - ‘ ‘ • - 0 (2.29) 

( - l)k  [ d2k/  dt2k  H^i  ] - 0 (2.30) 

Applying  these  equations,  we  consider  the  posetblllty  of  a singular 
pursuer  control  (a  similar  argument  holds  for  a singular  evader  control). 
The  first  necessary  condition  Is  from  Eq  (2.28), 

- 0 (D.l) 

Similarly,  all  time  derivatives  of  lp^,  are  rero,  so  that 

ip  „ - i'P  n - . . . - 0 (D.2) 

YP  yP 

From  the  Influence  function  equations  Eq  (5.20),  the  equation  for  X^  Is 
XP^  - Xpxp  vp  sin  / - XPyP  Vp  cos  y*’  (D.3) 

so  that  from  Eqs  (0.2)  and  (D.3) 

XPxp  sin  yP  - XPyP  cos  - 0 (D.A) 


f * *P„P  / *P 
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Taking  a time  derivative  of  both  aidea  of  Eq  (D.3)  and  aubetltutlng 
for  XPxp,  Xpyp,  Vp  and  yP  from  Eqs  (5.8)  and  (5.20)  ve  obtain  the 

equation  for  XPyP 

XPYp  - npg  (Vp  XPxP  coa  yP  + Vp  XpyP  aln  yp)  » 0 

“^P  (D.5) 

From  Eq  (P.5)  there  are  three  posaibilltlea 

(a)  XPxP  cos  Yp  + XPyp  sin  YP  - 0 
or 

(b)  np  - 0 


or 

(c)  both  (a)  and  (b) 

It  la  easy  to  establish  that  (a)  Is  not  possible  for  from  (a) 


tan  Yr 


- Xp  /Xp 
* yPM  *P 


vhich  Is  a contradiction  to  Eq  (D.4).  (b)  is  the  only  remaining 

possibility  so  that  on  a singular  surface  for  the  pursuer  the  equilibrium 
P* 

control  candidate  nr  Is 

np*  - 0 (D.6) 


A similar  result  holds  for  the  evaders  singular  control  if  it  occurs, 

ne*  - 0 (D.7) 

Application  of  the  necessary  condition  Eq  (2.30)  for  a singular  control 
for  the  pursuer  results  In  the  requlreaent 

Xpxp  cos  YP  + xpyp  sin  Yp  i 0 (D.8) 

Eq  (D.8)  must  be  satisfied  on  the  pursuer's  singular  surface.  Similarly 
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for  an  evader  singular  control,  the  inequality 
X*xC  cos  Y*  + ^Cye  sin  yC  ^ 0 

must  hold  on  the  evader's  singular  surface.  It  is 
that  Eqs  (D.8)  and  (D.9)  are  satisfied  with  strict 
terminal  singular  trajectories  for  the  pursuer  and 


(D.9) 

easily  verified 
Inequality  on 
evader  respectively. 


j 
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Appendix  E 

Trenevereellty  Conditions  for  the  Three  Pleyer  Problem 
In  this  sppendlx  the  vector  equation  lor  the  transversallty 
condltlona  In  the  three  player  problems  of  Chapter  V la  expanded  Into 
component  fora.  From  Eq  (2.25)  the  tranaversallty  condition  if 

a • a,  c,  d (E.l) 


A*  (t£)  - t ♦%  - (♦*/♦) 


1 


t-t. 


where  the  coat  functions  for  players  a,  c,  and  d are  (see  Eqs  (2.4)  and 
(4.4)) 


,c  .4 
J • J ■ t_ 


J*  - 1/2  ((x*-Xj)2  + (y'-yt>2l 


The  termination  criteria  from  Eq  (4.29)  la 

*Ix(tf),  tf)  - 1/2  [*dJ  (*CJ 

, „ ,,  d a.2  . , d a.2  .2. 

• ■1/2  [(x  -x  ) + (y  -y  ) - 1 ] 

,,  c a.2  . , c a.2  .2, 

[(x  -x  ) + (y  -y  ) - k ] 


(E.2) 

(E.3) 

(E.4) 


The  state  vector  x In  component  fora  Is 

T .dddcccaaa. 
x ■ (x  , y , y , x , y , y . * . y . Y ) 


(E.5) 


The  partial  and  total  derivatives  In  Eq  (B.l)  are 


♦c  - *d  .0 

x T x 


*\  - (0,  0,  0,  0.  0,  0.  x'-x^,  y»-yT,  0) 


(E.6) 

(E.7) 
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- V*  cos  ya  + (y*-yT)  Va  sin  ya 

where  f Is  defined  from  Che  state  equation, 

* (Vd  cos  yd , Vd  sin  yd , cd u , yc  cos  yC,  Vc  sin  y' 
Va  cos  ya,  Va  sin  ya, c av) 

Finally , 

+ - 1/2  I*11  i(ic  + *d  i|ic]  - ^ f 

■ (xd-xa)  (Vd  cos  yd-  Va  cos  ya)  d<c  + 

(yd-ya)  (Vd  sin  yd-  Va  sin  ya)  4>c  + 

(xC-xa)  (VC  cos  yC-  V8  cos  ya)  yd  + 

(yC-y“)  (Vc  sin  yc-  Va  sin  ya)  *d 
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* c *d 

The  equations  for  ♦ end  * are 

ife  • 2 (<xC-xa)  (VC  cos  YC  - v*  cos  y*) 

+(yC-ya)  (VC  Sin  Ye  - v*  sin  y“)1 
id  - 2 [(xd-x*>  (Vd  cos  Yd  - V*  cos  Y*>  + 

(yd-ya)  (Vd  sin  Yd  - v*  sin  y“> ) 

The  equation  for  q>  which  is  required  in  applying  -l'Hospital's 
rule  in  the  case  of  simultaneous  Intercept  is 

p * 1/2  l *d  *c  + i|id  pc  + d>c  + <|id  q»c  1 


In  the  case  of  simultaneous  intercept, 


pc  . *d  . o 


and  p in  Eq  (B.16)  becomes 

**  id 
* - * * 
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